EC203: Information, Meaning & Representation

Compression = Understanding

If you can compress it, you truly grok it

Coverage

• Bits, entropy, surprise — explained like psychology
• Encoding numbers, text, media; Unicode realities
• Compression as meaning extraction
• Ambiguity, context, semantics, grounding
• Symbolic vs distributed representations (LLMs)

Goals

• Speak about information precisely without jargon
• Design representations that serve your product + users
• Understand how LLM tokens, embeddings, vectors relate

Modules

1. Information Theory, but Friendly
2. Encoding Everything
3. Compression & Cognition
4. Semantics & Meaning-Making
5. Representations in AI Systems

Connections

• Builds on EC103; Required for EC401

Difficulty: ⭐⭐⭐⭐☆ | Time: 12 hours | Prereq: EC103