main page

Logging my dreams

I've been logging my dreams off and on. For a period of time it was handwritten little notebooks on my bedside table. Then back in 2013 I built a simple node-js web interface to store dreams in a mongolab database. I transcribed all those old dreams that I could still decipher, and started entering new ones directly on the website from then on.

The pandemic brought back renewed focus and I enhanced the interface with more tagging capabilities and a sentiment analysis feature.

Dream entry

Dreams are defined by a title, a date and a big text entry. They are tagged with keywords and people simply by using the # (for keywords) and @ (for people) signs inside the text. This then allows you to browse the dream database by hopping through recurring characters and themes.

Ambitiously I also added diary and dream-interpretation text fields, but they don't get much use.

Categorizing dreams

I experimented a bit with A.I. text analysis, hoping to find an automated way to define a dream's themes (content classification) and extract keywords, but I found the results too messy. Most likely due to the fact that a lot of my dream texts are rather short, and in typical dream-fashion rather chaotic in nature. In addition, writing them down while still half asleep, doesn't produce the best written English.

So I settled on a list of categories (or rather: not mutually exclusive content flags) that made the most sense to me. They have to be selected manually.

Most dreamed of

After entering a lot of dreams (I am at nearly 700 now) the simplest yet most informative feature of the database is a list of tags and characters sorted by occurrences. Naturally, what you dream of the most, is quite a tell.

But I have to say the most-occuring keywords are rather bland due to their mundanity. Words like 'phone' or 'office' or 'apartment' are pretty high up on that list, despite me not having obsessive dreams about phones or the office. There's clearly some adjustments to do with what content I decide to tag.

Sentiment analysis

Despite not being happy with the results of automated content classification of the dreams, I was still curious what other output Natural Language Processing tools could give me about my dreams.

I tried and compared a series of sentiment analysis tools (nlp.js, Compendium-js, WinkNLP, google's API, text2data). Some of them can do more than others but the main feature of 'sentiment analysis' is to assign positive or negative sentiment values to texts.

The one I ultimately implemented into the app is sentiment, a Node.js module that analyzes text based on the AFINN-165 dataset (3382 words coded with a goodness/badness value). After analysis of a text segment, the module hands back a score, a comparative value, and a list of those words it deemed as either good or bad.

The results seem overall ~ok, but they are definitely not perfect. Sometimes a text might contain a long list of deemed-negative words, yet still have an overly positive vibe. I assume these sentiment analysis tools are mainly used to judge the character of online feedback and reviews, and not inherently illogical and spasmodic content like dreams.

Graphing by comparative sentiment score

The graph on the left shows all my logged dreams with their comparative sentiment score (y-axis) across time (x-axis).

As you can see I've had spurts in my dream logging. One around 2003, then around 2011-2013, and then again since the start of the pandemic.

According to the y-axis distribution, it looks like my dreams lean more towards a slightly positive score.

It's then interesting to investigate the occasional outliers. That one dream with the -0.35 score for example mentions words like 'death', 'sad', 'trouble' but wasn't inherently a negative dream. And my most positive dream with a 0.27 score is a one-sentence text featuring the positive words 'nice' and 'dream'. So, questionable results. Which was definitely one of the reasons I decided to add 'stressful' and 'scary' as manual content flags.

But being able to sort my dreams by those sentiment scores is definitely an interesting way to browse through old dreams.

People on sentiment plot

An intriguing visualization is a plot showing each of your dream character's average sentiment scores. Who do you have the most positive or most emotionally charged dreams about?