token.h File Reference

Tokenization functionality. More...

#include <stddef.h>
#include <stdio.h>

Go to the source code of this file.

Data Structures
struct	as6502_token
	An individual token which will often be a part of a linked list of tokens. More...

Macros
#define	as6502_tokenIsEqualToStringLiteral(token, string)   as6502_tokenIsEqualToString(token, string, sizeof(string) - 1)
	Does a quick token comparison to a string literal.
#define	as6502_tokenListFindTokenLiteral(token, literal)   as6502_tokenListFindToken(token, literal, sizeof(literal) - 1)
	Searches through a linked list of tokens to find a token matching a given string literal.

Enumerations
enum	as6502_token_type { as6502_token_type_other, as6502_token_type_value }
	A simple enumeration type to differentiate values from other types of tokens. This could, in the future, be used for indicating other token types, such as assembler directives, instructions, etc.

Functions
as6502_token *	as6502_tokenCreate (const char *text, size_t loc, size_t len)
	Creates a single token object. Text is copied up to length indicated by len.
as6502_token *	as6502_tokenCopy (as6502_token *original)
	Creates a single token object that is identical to the passed in object, but in a different location. The next pointer is not copied.
void	as6502_tokenDestroy (as6502_token *token)
	Destroys a single token object.
void	as6502_tokenListDestroy (as6502_token *token)
	Destroys a token and all tokens attached to it by traversing the linked list of tokens.
int	as6502_tokenIsEqualToString (as6502_token *token, const char *string, size_t len)
	Does a quick token comparison to a string.
void	as6502_stringForTokenList (char *output, size_t len, as6502_token *head)
	Outputs a flat string for a given token list.
as6502_token *	as6502_firstTokenOfTypeInList (as6502_token *head, as6502_token_type type)
	Searches through a token list and returns the first token whose type matches the specified as6502_token_type.
size_t	as6502_lengthOfToken (const char *start, size_t len)
	Finds the first non-token character encountered, and returns it's location relative to the start pointer.
size_t	as6502_valueLengthInChars (const char *string, size_t len)
	Finds the first thing that isn't the value literal pointed to by string, and returns the length of the value literal.
as6502_token *	as6502_lex (const char *line, size_t len)
	Lexes a line of text into a linked list of tokens for later parsing.
as6502_token *	as6502_tokenListFindToken (as6502_token *token, const char *text, size_t len)
	Searches through a linked list of tokens to find a token matching a given string of a given length.
void	as6502_showDotForLinkedList (as6502_token *head)
	Generates and opens a full dot graph of a given list (This is a super convenience function for fully automatic dot graph display of an entire as6502_token linked list)
void	as6502_printDotForList (FILE *stream, as6502_token *head)
	Generates a full dot graph of a given linked list of as6502_token's (one entire source file)
void	as6502_printDotRankForList (FILE *stream, as6502_token *head)
	Generates the dot code for a given rank of as6502_token's (one line of assembly)

Detailed Description

Tokenization functionality.