The CPython interpreter really is an interpreter. But it also is a compiler. Python must go through a few stages before ever running the first line of code:

1. scanning
2. parsing

Older versions of Python added an additional stage:

1. scanning
2. parsing
3. checking for valid assignment targets

Let’s compare this to the stages of compiling a C program:

1. ~~preprocessing~~
2. lexical analysis (another term for “scanning”)
3. syntactic analysis (another term for “parsing”)
4. ~~semantic analysis~~
5. ~~linking~~

In this article you started implementing your own version of Python. To do so, you needed to create four main components:

A tokenizer: * accepts strings as input (supposedly, source code); * chunks the input into atomic pieces called tokens; * produces tokens regardless of their sequence making sense or not.

A parser: * accepts tokens as input; * consumes the tokens one at a time, while making sense they come in an order that makes sense; * produces a tree that represents the syntax of the original code.

A compiler: * accepts a tree as input; * traverses the tree to produce bytecode operations.

An interpreter: * accepts bytecode as input; * traverses the bytecode and performs the operation that each one represents; * uses a stack to help with the computations.

The tokenizer is the part of your program that accepts the source code and produces a linear sequence of tokens – bits of source code that you identify as being relevant.

• Tokenizer takes source code as input and produces tokens;
• Parser takes tokens as input and produces an AST;
• Compiler takes an AST as input and produces bytecode;
• Interpreter takes bytecode as input and produces program results.