Types of Tokens in C or C++ - Geeks4Coding

Types of Tokens

A token is the smallest element of a program that is meaningful to the compiler. Tokens can be classified as follows:

1.  Keywords
2. Identifiers
3. Constants
4. Strings
5. Special Symbols
6. Operators

Keyword:

 Keywords are pre-defined or reserved words in a programming language. Each keyword is meant to perform a specific function in a program. Since keywords are referred names for a compiler, they can’t be used as variable names because by doing so, we are trying to assign a new meaning to the keyword which is not allowed. You cannot redefine keywords. However, you can specify text to be substituted for keywords before compilation by using C/C++ preprocessor directives.C language supports 32 keywords which are given below:

auto             break         case            char            const            continue
default         do             double          else            enum            extern
float             for             goto              if               int                  long
register       return         short             signed        sizeof            static
struct           switch       typedef        union         unsigned        void
volatile        while

31 additional keywords other than C in C++ Keywords they are:


asm                bool                catch                 class
const_cast    delete            dynamic_cast   explicit
export            false               friend                 inline
mutable        namespace new                      operator
private           protected     public                 reinterpret_cast
static_cast    template       this                      throw
true                try                   typeid                typename
using             virtual            wchar_t

Identifiers: 

Identifiers are used as the general terminology for naming of variables, functions and arrays. These are user defined names consisting of arbitrarily long sequence of letters and digits with either a letter or the underscore(_) as a first character. Identifier names must differ in spelling and case from any keywords. You cannot use keywords as identifiers; they are reserved for special use. Once declared, you can use the identifier in later program statements to refer to the associated value. A special kind of identifier, called a statement label, can be used in goto statements.

There are certain rules that should be followed while naming c identifiers:

• They must begin with a letter or underscore(_).
• They must consist of only letters, digits, or underscore. No other special character is allowed.
• It should not be a keyword.
• It must not contain white space.
• It should be up to 31 characters long as only first 31 characters are significant.

Some examples of c identifiers:

 C program:

int main()
{
int a = 10;
}

In the above program there are 2 identifiers:

main: method name.
a: variable name.

Constants: 

Constants are also like normal variables. But, only difference is, their values can not be modified by the program once they are defined. Constants refer to fixed values. They are also called as literals.
Constants may belong to any of the data type.Syntax:

const

data_type variable_name; (or) const data_type *variable_name;

Types of Constants:

Integer constants – Example: 0, 1, 1218, 12482
Real or Floating point constants – Example: 0.0, 1203.03, 30486.184
Octal & Hexadecimal constants – Example: octal: (013 )8 = (11)10, Hexadecimal: (013)16 = (19)10
Character constants -Example: ‘a’, ‘A’, ‘z’
String constants -Example: “GeeksforGeeks”

Special Symbols: The following special symbols are used in C having some special meaning and thus, cannot be used for some other purpose.[] () {}, ; * = #

Brackets []: Opening and closing brackets are used as array element reference. These indicate single and multidimensional subscripts.

Parentheses (): These special symbols are used to indicate function calls and function parameters.

Braces {}: These opening and ending curly braces marks the start and end of a block of code containing more than one executable statement.

Comma (,): It is used to separate more than one statements like in for loop is separates initialization, condition and increment.

Semi colon: It is an operator that essentially invokes something called an initialization list.

Asterisk (*): It is used to create pointer variable.

Assignment operator: It is used to assign values.

Preprocessor (#): The preprocessor is a macro processor that is used automatically by the compiler to transform your program before actual compilation.

Operators: Operators are symbols that triggers an action when applied to C variables and other objects. The data items on which operators act upon are called operands. Depending on the number of operands that an operator can act upon, operators can be classified as follows:

Unary Operators: Those operators that require only single operand to act upon are known as unary operators.For Example increment and decrement operators

Binary Operators: Those operators that require two operands to act upon are called binary operators.

Binary operators are classified into :

• Arithmetic operators
• Relational Operators
• Logical Operators
• Assignment Operators
• Conditional Operators
•  Bitwise Operators

 

Ternary Operators:

 These operators requires three operands to act upon. For Example Conditional operator (?:).

A ternary operator has the following form,
exp₁ ? exp₂ : exp₃

The expression exp₁ will be evaluated always. Execution of exp₂ and exp₃ depends on the outcome of exp₁. If the outcome of exp₁ is non zero exp₂ will be evaluated, otherwise exp₃ will be evaluated.