Data and Visualization of Making My PONS Dictionary

• UUID: 96d8b873-3c90-4b96-9d83-0e5519a99f83

As mentioned in Making Better Dictionaries for Language Learning , I have been reading through and making my own digitalized dictionaries for several years now.

Proud and at the same time ashamed that I have finally finished making my digitalized dictionary of "PONS Basiswörterbuch Deutsch als Fremdsprache". Proud because I have finally completed the very early started project. Ashamed because I didn't manage it in an efficient way.

Nevertheless, besides the very valuable German language knowledge I have gained by reading through the dictionary, the process itself is also very much worthy looking back. Even just shallowly. By shallowly, I mean only descriptive data analysis is also interesting enough. (Although the charm of data analysis is not just descriptive but more about finding interesting patterns which could be of more values.)

What do I mean by "the process itself"? Well, it means several aspects. But here, in this article, I am targetting at the timestamps of individual words I have made the screenshots. These timestamps are also time points when I have read through a word of the PONS Basiswörterbuch.

Data Analysis of Screenshot Timestamps

One great aspect of digital data is that they carry timestamps when they get created or modified. So it is for all the screenshots I have made.

By using some python scripts, I gathered the timestamps data for all the screenshots into CSV files.

Based on the metadata I gathered, there are several apsects I can look into. For example, I know I mainly did the dictionary reading in relatively long time periods without breaking inbetween if I started reading on the corresponding day, there, it makes sense that I can group the timestamps based on the time interval from timestamp x and timestamp x - 1 to get a piece of output data as the following.

It would be too much to cover all in one article, therefore, in this article, I would like to narrow it down to just one focus - the timestamps (leaving the headwords aside).

This time series data is in fact quite special as it involves more than 1 continent which results in 3 time zones in total and the time points are very much unevenly distributed. So special care is needed to deal with the time values. By using some python scripts, the UTC timestamps can be turned into locale datetime strings as the following.

Data Visualization

Time series data is a very common type of data, therefore, luckily, there are already many great libraries well made for it.

Of course, data analysis is more about ideas first. So I started forming my visualization ideas on digital pen and paper. After several revisions, t he result is as the following.

Then it is about implementation with real data.

Data Visualization with matplotlib

Let's just see the result graphs with the code in it.

Matplotlib is great to make visualization only that it is too foundamental and therefore not so easy to use. For better looking graphs as well as the easy of date time handling, I went to Plotly.

Data Visualization with Plotly.py

What can I know from the graphs? At least that I know there were several times at midnight, at 0, I was still reading the dictionary and I know, I restarted the project several times.

Data Visualization with Plotly.js

And what's best about Plotly? It is powered by JavaScript, which makes it interactive and usable on the web pages.

Timestamp: added the following interactive visualization on 20240108

Visualize from another perspective - timestamps considered in daily chunks, and each day is further divided into subgroups - subgroups can be aggregated (summed) among different days. Further more, days can also be categorized into different groups based on days of the week or even take holidays into account.