Subtitle:
How modern AI represents knowledge, concepts, meaning, and reasoning inside neural networks.

Level
Beginner → AI Research Scientist

Duration
12 months

Goal

Understand what happens inside models like GPT, Gemini, Claude, DeepSeek, and other transformer-based systems—not by treating them as black boxes, but by studying the mathematics, geometry, and algorithms behind their internal representations.

Semester I — Foundations of AI Geometry

Module 1. What is Information?

Learn

• Information

• Signal

• Symbol

• Token

• Concept

• Meaning

• Context

• Compression

• Entropy

• Representation

Project

Represent one paragraph as:

• words

• tokens

• entities

• vectors

• graph

Module 2. Vector Spaces

Study

• dimensions

• coordinates

• basis vectors

• vector spaces

• manifolds

• latent space

• embedding space

• feature space

• semantic space

Explore how concepts become coordinates inside AI.

Module 3. Linear Algebra for AI

Study

• vectors

• matrices

• tensors

• projections

• transformations

• eigenvectors

• eigenvalues

• singular value decomposition

A central concept is how vectors combine to represent meaning.

Module 4. High-dimensional Geometry

Topics

• 2D

• 3D

• 100D

• 768D

• 1536D

• 4096D

• curse of dimensionality

• neighborhoods

• manifolds

• topology

Project

Visualize embeddings using

• PCA

• t-SNE

• UMAP

Semester II — Embeddings

Module 5. Embedding Models

Study

• Word2Vec

• GloVe

• FastText

• BERT embeddings

• OpenAI embeddings

• multilingual embeddings

Understand

How "king", "queen", "Paris", and "dog" become vectors.

Module 6. Semantic Coordinates

Topics

• semantic similarity

• latent variables

• conceptual distance

• analogies

• vector arithmetic

• semantic interpolation

Examples

King − Man + Woman ≈ Queen

Projects

Create your own embedding model.

Module 7. Measuring Meaning

Learn

• cosine similarity

• Euclidean distance

• Manhattan distance

• angular distance

• dot product

Study why nearby vectors usually represent related meanings.

Module 8. Vector Databases

Study

• ANN

• HNSW

• IVF

• Product Quantization

• FAISS

• ScaNN

Systems

• Qdrant

• Milvus

• Weaviate

• Pinecone

• Chroma

Projects

Build semantic search.

Semester III — Inside Large Language Models

Module 9. Tokenization

Learn

• Byte Pair Encoding

• SentencePiece

• token IDs

• vocabulary

• subwords

Understand why AI sees tokens—not words.

Module 10. Transformer Architecture

Study

• embeddings

• positional encoding

• attention

• multi-head attention

• feed-forward networks

• normalization

• residual connections

Projects

Build a miniature transformer.

Module 11. Attention

Topics

• Query

• Key

• Value

• self-attention

• cross-attention

• attention heads

• attention maps

Understand how AI determines which information matters most in context.

Module 12. Internal Representations

Study

• hidden states

• activations

• neurons

• circuits

• features

• representations

• abstraction levels

Research

What does one neuron represent?

Semester IV — How AI "Thinks"

Modern AI does not "think" like a human with explicit symbolic reasoning. Instead, it performs repeated transformations of numerical representations that encode statistical and semantic patterns learned from data.

Module 13. Reasoning in Neural Networks

Study

• inference

• chain of computation

• reasoning traces

• internal planning

• world representations

• latent reasoning

Module 14. Memory

Learn

• context window

• KV cache

• long-term memory

• retrieval memory

• graph memory

• episodic memory

• semantic memory

Module 15. Mechanistic Interpretability

Research

• feature visualization

• sparse autoencoders

• activation steering

• neuron analysis

• superposition

• circuits

• safety

Projects

Interpret a small transformer layer by layer.

Module 16. Toward AI Cognitive Architectures

Study

• symbolic reasoning

• neural reasoning

• hybrid systems

• GraphRAG

• world models

• planning

• agent architectures

• multimodal representations

• embodied AI

Mathematics

• Linear Algebra

• Calculus

• Probability

• Statistics

• Information Theory

• Optimization

• Numerical Methods

• Geometry

• Topology

Programming

• Python

• NumPy

• PyTorch

• JAX

• CUDA (optional)

• C++

• Rust (optional)

Libraries

• Transformers

• PyTorch

• TensorFlow

• Hugging Face

• FAISS

• Sentence Transformers

• NetworkX

• Qdrant Client

• LangChain

• LlamaIndex

Research Topics

• Representation Learning

• Latent Spaces

• Embedding Alignment

• Vector Compression

• Multimodal Embeddings

• Sparse Representations

• Mechanistic Interpretability

• World Models

• Neural Memory

• AI Reasoning

• Graph + Vector Hybrid Systems

• Neuro-symbolic AI

Final Research Project

Design and implement an AI Cognitive Engine that combines:

• tokenization

• embedding generation

• latent semantic spaces

• vector databases

• knowledge graphs

• GraphRAG

• transformer inference

• attention visualization

• mechanistic interpretability tools

• long-term memory

• autonomous reasoning

• AI agents

vocabulary 

Level 1. Reality

The world exists independently of any observer.

Reality

Everything that exists or occurs, regardless of whether it is observed or described.

Level 2. Object

An identifiable part of reality.

Object

Anything that can be distinguished from everything else.

Examples

• a tree

• a person

• an atom

• a company

• a galaxy

Level 3. Observation

Reality becomes observable.

Observation

The act of detecting or measuring some aspect of an object or event.

Level 4. Data

The first digital layer.

Data

Raw recorded values without guaranteed interpretation.

Examples

42

John

101010

25.4

2026-07-19

Data answers

What was recorded?

Level 5. Signal

Information carried through change.

Signal

A measurable variation that can convey data.

Examples

• sound

• light

• voltage

• radio waves

• pixels

Level 6. Symbol

A representation.

Symbol

Something that stands for something else.

Examples

A

π

❤️

123

cat

Level 7. Token

What AI actually reads.

Token

The smallest unit processed by a language model.

Examples

Knowledge

Know

ledge

Graph

Tokens are computational units rather than linguistic ones.

Level 8. Information

Data interpreted within context.

Information

Data that reduces uncertainty for a receiver.

Example

Temperature = 38°C

Without context

→ data

With context

→ information

Level 9. Meaning

The interpretation.

Meaning

What information refers to in reality or within a conceptual system.

Level 10. Semantics

The science of meaning.

Semantics

The relationships between symbols and what they represent.

Example

Jaguar

Possible meanings

• animal

• automobile brand

• software project

Semantics resolves ambiguity.

Level 11. Context

The hidden information.

Context

Additional information required for correct interpretation.

Example

Apple

Technology

↓

Company

Fruit

↓

Food

Context determines meaning.

Level 12. Entity

A uniquely identifiable thing.

Entity

An object, concept, event, place, organization, or person with its own identity.

Examples

• Albert Einstein

• Earth

• Microsoft

• Gravity

• World War II

Entities become nodes inside knowledge graphs.

Level 13. Attribute

A property.

Attribute

A characteristic describing an entity.

Example

Person

Attributes

• name

• age

• nationality

• height

Level 14. Relation

A connection.

Relation

A meaningful link between entities.

Examples

works_for

part_of

located_in

created_by

owns

friend_of

Level 15. Fact

Verified knowledge.

Fact

A relationship or statement accepted as true within a given context.

Example

Earth orbits the Sun.

Level 16. Knowledge

Organized understanding.

Knowledge

A structured network of interconnected facts, concepts, relationships, and models that can support reasoning and decision-making.

Knowledge is not simply a collection of facts—it includes how those facts relate to one another.

Level 17. Model

A simplified representation.

Model

A representation of reality created to explain, predict, or simulate phenomena.

Examples

• climate model

• language model

• economic model

Level 18. Representation

How something is encoded.

Representation

The form in which information or knowledge is stored.

Examples

• text

• table

• graph

• vector

• image

• audio

• tensor

Level 19. Embedding

Semantic coordinates.

Embedding

A numerical vector that represents the meaning or properties of an object in a high-dimensional space.

Example

Dog

↓

[-0.82, 0.34, 1.25, ...]

Nearby vectors usually correspond to semantically similar concepts.

Level 20. Vector

Coordinates.

Vector

An ordered list of numbers representing a position or direction in a mathematical space.

For AI, vectors encode concepts rather than physical locations.

Level 21. Latent Space

The hidden conceptual world.

Latent Space

A high-dimensional space in which AI organizes representations according to learned patterns and similarities.

Nearby regions tend to correspond to related concepts, even if the relationships were never explicitly programmed.

Level 22. Knowledge Graph

Structured knowledge.

Knowledge Graph

A network of entities connected by explicitly defined relationships.

Example

Einstein

     │

 born_in

     │

Ulm

     │

located_in

     │

Germany

Level 23. Ontology

The blueprint.

Ontology

A formal specification of the concepts, categories, properties, and relationships within a domain.

It defines what kinds of entities exist and how they can be related.

Level 24. Semantic Density

Meaning per representation.

Semantic Density

The amount of meaning conveyed relative to the size of its representation.

Examples

Tesla

may convey far more meaning than a long descriptive sentence if the audience shares the necessary background knowledge.

Higher semantic density means richer meaning with fewer symbols, provided comprehension is preserved.

Level 25. Semantic Compression

Compression without losing meaning.

Semantic Compression

Reducing the size of a representation while preserving its essential meaning.

Unlike traditional data compression, semantic compression relies on shared knowledge and conceptual abstractions.

Level 26. Intelligence

Adaptive understanding.

Intelligence

The ability to acquire knowledge, build internal models, reason about situations, learn from experience, and apply understanding to achieve goals across different contexts.

The complete hierarchy

Reality

    ↓

Object

    ↓

Observation

    ↓

Signal

    ↓

Data

    ↓

Symbol

    ↓

Token

    ↓

Information

    ↓

Meaning

    ↓

Semantics

    ↓

Context

    ↓

Entity

    ↓

Attribute

    ↓

Relation

    ↓

Fact

    ↓

Knowledge

    ↓

Model

    ↓

Representation

    ↓

Embedding

    ↓

Vector

    ↓

Latent Space

    ↓

Knowledge Graph

    ↓

Ontology

    ↓

Semantic Density

    ↓

Semantic Compression

    ↓

Intelligence