ANLP_WS24_CA1/README.md

ANLP_WS24_CA1

Master MDS

Use NLP techniques you learned so far (N-gram models, basic machine learning, no neural nets) to analyse texts or to build an application. Document your approach.

Data Source

https://github.com/taivop/joke-dataset/tree/master

File	Jokes	Tokens
reddit_jokes.json	195K jokes	7.40M tokens
stupidstuff.json	3.77K jokes	396K tokens
wocka.json	10.0K jokes	1.11M tokens
TOTAL	208K jokes	8.91M tokens

Topic presentations (graded) (5 min)

Focus:

• What is your overall idea?
• What kind of data will you use and where do you get the data?
• Your approach, which techniques will you use?
• Expected results.

Open Questions:

• How to evaluate similarity?
• How to find structural patterns? (like phrases, setups, punchlines, or wordplay)

Possible Hypothesis:

• Similar jokes share more common n-grams, phrases, or structural patterns.
• Basic features like word frequency, sentiment, length, or punctuation can predict joke ratings.

other ideas:

• The length of a joke (measured in words or characters) is inversely correlated with its average rating, as shortness may enhance comedic impact.

• Highly rated jokes follow certain structural patterns (e.g., setups, punchlines, or wordplay).

Possible Tools / Techniques

• Text Preprocessing: Tokenization, stopword removal, stemming/lemmatization.

• Feature Extraction: Bag-of-Words, n-grams (bigram/trigram analysis), TF-IDF.

• Similarity: Cosine similarity for finding similar jokes.