NLP Concept Cards

Lesson 10: How AI Understands Language - Natural Language Processing

COLOR VERSION

✂️
Tokenization
Definition: The process of breaking text into individual words or meaningful units (tokens). Why It Matters: First step in text analysis. Computers must identify where one word ends and another begins. Key Challenge: Handling contractions, hyphenated words, punctuation, and numbers.
💬
Tokenization
Example:

"I love AI!"

Becomes:
  • "I"
  • "love"
  • "AI"
  • "!"

4 tokens total

🏷️
Part-of-Speech Tagging
Definition: Labeling each word by its grammatical function: noun, verb, adjective, adverb, etc. Why It Matters: Helps AI understand sentence structure and relationships between words. Key Challenge: Same word can be different parts of speech depending on context.
📝
POS Tagging
Example:

"The quick brown fox jumps"

  • The = Determiner
  • quick = Adjective
  • brown = Adjective
  • fox = Noun
  • jumps = Verb
🎯
Named Entity Recognition
Definition: Identifying and categorizing key information: people, places, organizations, dates, money, quantities. Why It Matters: Extracts important facts from text automatically. Key Challenge: Distinguishing between common words and names (e.g., "Apple" the fruit vs. Apple Inc.).
📍
Named Entity Recognition
Example:

"Steve Jobs founded Apple in California in 1976."

  • Steve Jobs = PERSON
  • Apple = ORGANIZATION
  • California = LOCATION
  • 1976 = DATE
😊
Sentiment Analysis
Definition: Determining whether text expresses positive, negative, or neutral emotions. Why It Matters: Companies track customer satisfaction, monitor social media, detect harmful content. Key Challenge: Sarcasm, mixed emotions, and cultural context are hard to detect.
💭
Sentiment Analysis
Examples:
  • "This pizza is amazing!" = POSITIVE 😊
  • "Worst movie ever." = NEGATIVE 😠
  • "The store opens at 9am." = NEUTRAL 😐
  • "Oh great, more rain." = NEGATIVE (sarcasm) 🙄
💡
Semantic Analysis
Definition: Understanding the actual meaning and relationships between words in context. Why It Matters: Goes beyond individual words to grasp overall meaning, intent, and implications. Key Challenge: Requires understanding context, idioms, implied meaning, and common sense.
🧠
Semantic Analysis
Example of Ambiguity:

"Can you pass the salt?"

Not literally asking about ability!

It's a polite request for action. Semantic analysis helps AI understand this is a command, not a question about capability.

🤖
Application: Chatbots
How NLP is Used:
  • Tokenize user messages
  • Identify intent (what user wants)
  • Extract key entities (names, dates)
  • Generate appropriate responses
  • Maintain conversation context

Examples: Customer service bots, virtual assistants, tutoring chatbots

🌍
Application: Translation
How NLP is Used:
  • Analyze source language structure
  • Identify words and their meanings
  • Find equivalent words in target language
  • Reconstruct sentence with proper grammar
  • Adjust for cultural context

Examples: Google Translate, language learning apps, document translation

🗣️
Application: Voice Assistants
How NLP is Used:
  • Convert speech to text
  • Understand spoken commands
  • Extract action requests and parameters
  • Generate natural language responses
  • Convert responses to speech

Examples: Siri, Alexa, Google Assistant

🛡️
Application: Content Moderation
How NLP is Used:
  • Analyze text for harmful keywords
  • Detect aggressive sentiment
  • Identify hate speech patterns
  • Flag potentially dangerous content
  • Reduce spam and scams

Examples: Social media platforms, comment sections, email filters

Challenge: Ambiguity
The Problem: Words and sentences can have multiple meanings depending on context. Example:

"I saw her duck"
• Her pet duck?
• She ducked down?

Why It's Hard: Humans use context clues and common sense automatically. Computers need explicit rules.
🙄
Challenge: Sarcasm
The Problem: Sarcasm means the opposite of what's literally said, often for humorous or critical effect. Example:

"Oh wonderful, another pop quiz!"

Contains positive word "wonderful" but speaker is unhappy.

Why It's Hard: Requires understanding tone, context, and social cues that computers can't detect in text.
🤔
Challenge: Idioms
The Problem: Idioms and expressions don't mean what their words literally say. Examples:
  • "Break a leg" = Good luck
  • "It's raining cats and dogs" = Heavy rain
  • "Piece of cake" = Very easy
Why It's Hard: Cultural knowledge required. Different languages have different idioms.
🔄
Challenge: Context Dependence
The Problem: Word meanings change based on surrounding context and previous conversation. Example:

"The bank is closed."

Financial bank? River bank?
Need context to know!

Why It's Hard: Computers must track long conversations and understand how context affects meaning.
📱
Real-World: Autocorrect
How It Works:
  • Analyzes what you type
  • Compares to dictionary of words
  • Predicts intended word using context
  • Suggests corrections for typos
  • Learns your common phrases
NLP Concepts Used: Tokenization, spelling correction, predictive text, contextual understanding
📧
Real-World: Spam Filters
How It Works:
  • Analyzes email subject and content
  • Identifies suspicious keywords
  • Checks sender reputation
  • Detects phishing patterns
  • Learns from user feedback
NLP Concepts Used: Text classification, pattern recognition, sentiment analysis, entity recognition
🎬
Real-World: Review Analysis
How It Works:
  • Collects thousands of customer reviews
  • Analyzes overall sentiment (positive/negative)
  • Identifies common complaints or praise
  • Summarizes key themes
  • Tracks sentiment over time
NLP Concepts Used: Sentiment analysis, key phrase extraction, topic modeling, aggregation
🔍
Real-World: Search Engines
How It Works:
  • Understands your search query intent
  • Identifies key terms and entities
  • Finds semantically related content
  • Ranks results by relevance
  • Suggests related searches
NLP Concepts Used: Query understanding, semantic search, entity recognition, ranking algorithms
💬
Discussion Question
Group Discussion:

"If you could design an NLP-powered tool to solve a problem in your daily life, what would it do? Which NLP concepts would it need to use?"

Think about:
  • What problem does it solve?
  • Who would use it?
  • What challenges might it face?
🤝
Discussion Question
Group Discussion:

"What is the biggest challenge computers face in understanding human language? How might AI overcome this challenge in the future?"

Consider:
  • What makes language hard for computers?
  • What do humans do naturally that's hard to program?
  • How could technology improve?

© 2025 Evolve AI Institute LLC
Free AI Education for All
Visit: evolveaiinstitute.com/edai