Posts | Thought splinters

Introduction to R for Social Scientists (I2RSS)

Sat, 02 Apr 2022 00:00:00 +0000

Acclamations for I2RSS

From my training (Ph.D. in sociology) but also from my different professorship on educational sciences, I was pleased finally to see a new R introduction especially directed at social scientists. The book title “Introduction to R for Social Scientists: A Tidy Programming Approach” (abbreviated I2RSS) was promising for me for two reasons.

The book is written by Ryan Kennedy and Philip Waggoner, two prestigious scientists involved in political and social research. Whereas Ryan has an astonishing career in political science, is Philip exceptionally experienced at an academic level in computational social science and data science. Philip is also a member of easystats (a software group that tends to an ecosystem of packages making statistics in R easy).

Promoting the modern tidyverse approach

The book features the tidyverse approach a modern collection of R packages that share an underlying design philosophy, grammar, and data structures.

Real data, results discussed and gentle dialog with reader

Without doubts, the book has several other merits: It uses real data sets, e.g., from the American National Election Studies (ANES) and teaches data management, visualizations, exploratory data analysis (EDA), and statistical modeling not in an abstract way, but also discusses the exciting results with their data sets. It talks to the reader in a very sympathetic and comforting way to master the (still very) steep learning curve for R beginners.

Some drawbacks and failures

But unfortunately, there are also two significant drawbacks to this introduction:

Educational inconsistencies

The book has several weaknesses from an educational point of view:

With some exceptions (data classes, data structures, and the map function from the purrr package is the chapter on “Essential programming,” in my personal view to advanced for R beginners, especially for people from the social sciences with no programming background.
Comparing the same results with base R and tidyverse code may be interesting for intermediate users. Still, it is a waste of time and complicates things for beginners. It is also misusing the sparse place in the book, which has only 198 pages.
The book uses R scripts and does not feature Rmd files. This is strange as literary programming is an essential step for reproducibility, and RMarkdown also facilitates interactive use of the program code.
Sometimes, the results of the code chunks are missings (e.g., graphs, tables, or reports). The explaining text under the code is, therefore, abstract. There are also some link errors, providing an URL starting with “www” where the referring webpage does not need/accept this addition.

These above weaknesses result in the deduction of one of the five stars.

Not a role model for Open Science

The second star deduction is motivated by missing teaching modern scientific behavior for early career researchers (ECRs). This shortcoming includes the prominent use of R Markdown and the use of collaboration and Open Science. I think that after the publication of Happy Git, there is no excuse anymore to teach Git and GitHub (or similar systems) even for R beginners. Another big mistake is not using the open-access politics of the bookdown.org infrastructure for the online version of the book.

In the book, the authors promote the friendliness and importance of the R community several times. They even mention that the book was initially written with the help of the bookdown package (page 2). So why not supply a role model for R beginners and young researchers?

repairData - My First R Package at CRAN

Sun, 24 Oct 2021 00:00:00 +0000

R Packages book as a valuable assistance

I am pleased: repairData is on CRAN (The Comprehensive R Archive Network! repairData is my first package; offers only five data sets and is therefore not a brilliant feat. The real achievement for me was that I successfully completed the complex submission process for the first time. Without R Packages, the still not finished but already publicly accessible 2nd edition by Hadley Wickham and Jenny Bryan, I wouldn’t have succeeded – at least not relatively easy on my second trial.

Book Cover of ‘R Packages’

Goal of the datasets

repairData offers the dataset of the Open Repair Alliance (ORA), from February 21, 2021. The data set corresponds to the Open Repair Data Standards (ORDS) developed by the ORA in its last version, 0.2.1.

Data from five providers can currently downloaded separately from the Open Repair Alliance (ORA) website. When I developed repairData, I thought that it would be an advantage for users if they had the data already aggregated and available. Users can separate the different databases again via the “Data provider ID” category.

On the GitHub repository of the Open Repair Alliance, I also found four other data sets on batteries, printers, smartphones, and tablets. Data sets with the corresponding categories were extracted from the database and examined separately as part of a concerted online campaign.

The main aim of these excerpts is to find out the causes of the defects and what obstacles arose during the repair. At the repair events, there is usually not enough time to investigate such more targeted questions. In principle, these database extracts differ from the primary data record only by an additional column to categorize the errors.

The goal of the various initiatives in the ORA is to collect repair data in a standardized manner and make them publicly available. The dataset should provide evidence of shortcomings in our affluent society: Planned obsolescence, wasted energy, filigree construction, and limited access to repairs. The collected data should help build up pressure against companies and the government for changes in production and regulations.

Five data providers

The five organizations of the ORA are:

Repair Café International

The Dutch journalist Martine Postma started the Repair Café movement by establishing the first Repair Café on October 18, 2009, in Amsterdam. On March 2, 2010, the Repair Café International Foundation was founded as an umbrella organization based in Amsterdam. There are now 2196 members (Repair Cafés) in this network worldwide. 23,048 data sets, almost half of all data, come from this network alone.

Google map with global distribution of 2.196 repair cafés (October 23, 2021).

Community repair data is collected for the Repair Café International foundation via the RepairMonitor platform. The RepairMonitor is the online tool for collecting and sharing repair data. The Repair Café International website has an excellent 2-minute video that explains repair customers' processes in the repair cafés.

anstiftung

anstiftung is a non-profit foundation based in Munich that, in addition to repair initiatives, also promotes other movements to revitalize neighborhoods or interventions in public spaces, such as intercultural and urban gardens, open workshops, and open source projects.

anstiftung is also active as a funding company. It researches, publishes, and organizes the independent German network of over 1,000 repair initiatives. Repair cafés are held online continuously (weekly!)

Collection of different posters for anstiftung events.

The community repair data will be collected in the repair initiatives platform from June 2018 (see the repo on GitHub). anstiftung has 4,798 records to the collective database contributed so far.

Screenshot of startpage of the reparatur-initiativen network.

Fixit Clinic

Fixit Clinic is a US initiative that “got out of hand as a hobby” (founder Peter Mui in a YouTube video (00: 21-00: 24)). Fixit Clinic is a “pop-up” activity where people take their broken things apart. The Fixit Clinic provides coaches who guide the participants through the repair process and provide the special tools for the repair.

In addition, Fixit Clinic is known worldwide for its ifixit website, which provides “repair instructions for everything, written by everyone”. The website is an extensive collection of advice and community forums in several languages and offers an online shop for repair tools.

Screenshot of fixit website.

The data is collected using the Broken Item Report form as part of the Fixit Clinic’s community repair events. In addition to the usual information (article, brand, model, year of construction, problem), the form also asks other questions about the defective device, such as information on the exchange of repair experience, the status of the spare parts, and what experiences there were with the support of the manufacturer. The form is a kind of “routing slip” with a constantly updated status report on the item. Both – successful and unsuccessful –repair attempts at the events are entered. Even the owners document their experiences of the daily uses. Fixit Clinic has so far participated in the shared database with 487 data records.

Screenshot of ifixit webpage with link to the broken item report.

Repair Café Wales

Repair Café Wales opens and supports repair cafés across Wales. Unlike the other grassroots initiatives, this project is funded by the Welsh Government (Welsh Government’s Landfill Disposal Tax Community Scheme) and administered by the Wales Council for Voluntary Action.

Screenshot of website of the Repair Café Wales.

Customers fill out a form after arriving at the repair café. There are different forms to access depending on the repair item: Apparel/Textiles, Bag/Luggage/Purse, Bicycle, Bicycle Accessory, Car Accessory, Furniture, Heater, Home Decor, Jewelry/Accessory, Manual Tool, Misc, Scooter, Electrical Item. As far as I can see from the first page, only the form for the electrical items is different, sporting a list of categories.

So far, 2505 records are documented via the Repair Cafés Wales.

The Restart Project

The Restart Project is a London-based social enterprise. It organizes regular restart parties where participants teach each other how to repair or get their broken or slow devices back on the road. The Restart project specializes in electrical appliances, with the joint training and repair campaign ranging from toasters to headphones to iPhones and tablets.

Screenshot of homepage of The Restart Project, ‘a people-powered social enterprise that aims to fix our relationship with electronics’.

The Restart Project also works with schools and other organizations and is co-founder of the European Right to Repair Campaign. This global movement advocates that everyone has the right to repair the products they own themselves¹. The move aims to change the rules for making the devices so that they are easy and affordable to repair. Another concern of Restart is to protect or expand the owners' rights after the purchase and to combat the planned obsolescence².

Screenshot of the ‘Right to Repair Initiative’ homepage.

Data is collected at community repair events and recorded in the Restarters.net Fixometer module. Fixometer is the software and database for recording the effects of community repair activities (see also the repository on GitHub). The Restart Project is known for its scientifically-based calculation of the CO2 savings by extending the life cycles of the repaired devices (see the FAQ on the three topics:

How is data collected? How are the calculations made? What can we learn from the data?

Banner of the Restarters website, which compiles and continuously updates the successes of the repair efforts.

Due to their strict methodical approach, only the organizers of the restart parties log the item’s status of the appliances brought to the event for repair. The organizers are also the ones who then enter the results in the Fixometer online database. The Restart Project has already contributed 17,840 data records to the joint data acquisition.

Aim of repairData package

With the data preparation for the large R community, I hope that the data set will attract more attention and be examined more closely by statistics, visualization, and modeling experts. I myself also plan to take a closer look at the data as part of a planned R tutorial project.

Screenshot of the repairData documentation page, written with the pkgdown package.

“We are people, we are repairers, we are sustainability activists, we are community. —— We believe products should last longer, and therefore when broken, they should be repaired. This requires products to be designed for repair as well as support for repairers of all kinds. —— We are asking for the right to repair." ↩︎
We talk about planned obsolescence when a manufacturer consciously makes design and manufacturing decisions that results in a limited product life. This doesn’t necessarily mean that products are made to break after a certain period. If a manufacturer designs a product in such a way that it cannot be disassembled, or if they decide not to make spare parts, repair information, or software updates available, they are making a deliberate choice that result in a limited product life – the product becomes obsolete due to the manufacturer’s decision not to support its prolonged use. ↩︎

Using GitHub as a Mentoring Tool

Sun, 01 Aug 2021 00:00:00 +0000

Beginner, Intermediate, and Expert Level

For several years I am interested to learn R, the free software environment for statistical computing and graphics. After

reading several books,
finished some (paid) introductory (MOOC) courses successfully with Coursera
and especially some smaller (paid) projects

I would say that I am now (stuck?) on an intermediate skill level. I believe that this is the most challenging step in progressing to an expert for a self-determined learner. There are always so many different avenues to follow up. This is especially true with R and its

ever increasing amount of new R packages (To date: 20010 (MRAN) resp. 17954 (CRAN), 3452 (Bioconductor).
hundreds of books (see: the searchable list on r-project.org, a curated list on GitHub by Roman Tselgelskyi, but also my Wakelet on free R Books
an unknown number of (free) online tutorials (see the general list and university listed free R tutorials by Pairach Piboonrungroj and my Wakelet on learn R programming)

Comparing my skill level with the three learner paths suggested by RStudio Education, I could affirm my personal assessment. From the six tasks proposals on the intermediate path

grab some R cheat sheets
learn to get help (see, for instance, my activities on StackOverflow)
improve your visualizations
develop interactive applications with htmlwidgets and Shiny
simplify your model explorations with tidymodels
explore other specialized packages

I have some skills with the first three, and I also experimented already with Shiny. I even had worked on some steps at the expert levels by writing an R package: (bib2academic got 16 stars, but I did not manage to submit it to CRAN. It is now obsolete as a similar function was integrated into the Academic Hugo website (now Wowchemy website builder).

Project for Motivation Necessary

But to continue this path, I would need some strong motivation. I tried, for instance, to read the (not yet finished) book Mastering Shiny. As with all books and papers by Hadley Wickham, it is well written and exciting. But to learn continuously, I would need a practical project where I get not only real-world challenges but also a strong personal motivation to complete the product. In finishing the project/product, I would have to learn to overcome several real-world challenges where I would need to look for help and consult/experiment with several new functions or packages.

But this strategy has two advantages:

It is very laborious as the specific questions are not (yet) clear enough. Fora like StackOverflow are not suitable as the problem space is not delimited, a concrete coding question with a REPREX (REPRoducible EXamples) is not available.
It is error-prone and not efficient. Am I posing the right question? Do I look at the right places? It is not guaranteed that one finds the optimal solution. This is particularly awkward if the answer in the R community is very well known.

Looking for a Human Mentor

I believe with a mentor, one could overcome these insecurities fast. The mentor — an experienced user — could ask questions to narrow the search space and point out helpful hints about what and where to look. Wikipedia, for instance, has established a similar program for their user community called Adopt-a-user.

The Adopt-a-user program is designed to help new and inexperienced users by pairing them with more experienced Wikipedians. These editors (referred to as adopters or mentors) will “adopt” newer users, guiding them along the way as they learn about Wikipedia and its various aspects. (Wikipedia)

In Wikipedia, the intention is to help novice contributors get orientation and some guided practice with the very complex rule set for writing or changing Wikipedia entries. My thought is to find a mentor to advance at the intermediate level. I think there is plenty of material for the R beginners freely available. But more important: You will learn R from scratch as an individual and do not need to worry at the beginning stage about an already developed complex rule which you have to observe and follow.

As far as I know, there is no mentor program for R users. The next best alternative is a more open and friendly community. An excellent example, in my opinion, would be the RStudio Community. But even there, you should have a concrete question to ask.

GitHub as a Mentoring Tool

An instructive example

I recently found out that GitHub could be used as an educational tool. I will you give an example which was very exciting for me:

Often I was confronted with the problem of how to calculate and format nicely a contingency table in R. The free web material on the web on this question is abound (e.g., Datacamp, Data Science Central, R-Bloggers, TechVidvan, Statology, …). But many examples still teach the Base R commands. They do not mention new procedures with the tidydata approach and related packages like tidyr. And they do not worry about outputting the table in a publishing-ready format.

There are many contingency tables published in the recently finished books Introduction to Modern Statistics.

Table 1: Descriptive statistics for the stent study.
	30 days	365 days
Group	Stroke	No event	Stroke	No event
Control	13	214	28	199
Treatment	33	191	45	179
Total	46	405	73	378

I could look up the source code from lines 85-101 via GitHub and learn how to produce the above table. (In the following code chunk, I have focussed on the table appearance and left out different previous data transformations to prepare the two data sets stent30 and stent365. Furthermore, the code differs somewhat from the original due to some CSS styles of this website.).

```{r stent-study}
suppressMessages(library(janitor))
suppressMessages(library(kableExtra))
suppressMessages(library(openintro))
suppressMessages(library(tidyverse))

stent %>%
  mutate(group = str_to_title(group)) %>%
  pivot_longer(cols = c(`30 days`, `365 days`), 
               names_to = "stage", 
               values_to = "outcome") %>%
  count(group, stage, outcome) %>%
  pivot_wider(names_from = c(stage, outcome), values_from = n) %>%
  adorn_totals(where = "row") %>%
  kbl(linesep = "", booktabs = TRUE, caption = "Descriptive statistics for the stent study.",
      col.names = c("Group", "Stroke", "No event", "Stroke", "No event"),
      table.attr = "style='width: 75%;'") %>%
  add_header_above(c(" " = 1, "30 days" = 2, "365 days" = 2), extra_css = "border-bottom: 2px solid") %>%
  row_spec(1, extra_css = "border-top: 2px solid") %>%
  row_spec(3, extra_css = "border-top: 2px solid") %>%
  kable_styling(bootstrap_options = c("striped", "condensed"), 
                latex_options = c("striped", "hold_position"),
                full_width = T, position = 'center')
```

Lesson learned

Looking into the source code at GitHub, I learned several issues:

I experimented already with the janitor package, and I am happy to get the confirmation that it is a widely used package to “provide quick counts of variable combinations (i.e., frequency tables and crosstabs)” and to “format [nicely] the tabulation results.” (From the Janitor package description.)
I also did know about the kableExtra package and had the kable_styling function already used several times.
The other kableExtra functions (add_header_above(), row_spec(), kbl()) I had not used before. Especially add_header_above() has an important role for special headings in contingency tables.
The parameter extra_css was entirely new for me. I have never heard about it and didn’t even know that it exists.

For me, this learning experience was very instructive! I admit that this example could have been asked via a friendly forum: For instance: “How could I produce a contingency table formatted professionally for print by using the tidyverse approach?” Additionally, one would have to provide some code to show where one stands and what approach one is using.

Looking Experts over their Shoulders

But this was just one example where I knew about the problem and had already tried several approaches to find a solution. But what about code patterns I didn’t even know and could therefore not ask the right question?

Finding a GitHub project with code snippets one needs urgently is just the beginning. There is a big chance that the same repository (aka ‘repo’) hides more code snippets to learn from. And maybe the person behind this one GitHub repos maintains other repositories that are also very interesting?

In my case, it turned out that Mine Çetakaya-Rundel, one of the authors and the person responsible for the repository management, has educational approaches that are similar to mine. But she is way off more skilled in R and statistics. I am especially interested in her usage of the learnr package for interactive tutorials and have watched a series of three videos on Teaching Statistics and Data Science Online.

In a certain sense, she functions as a mentor for me — even if she doesn’t know about it 🥸. Meanwhile, we had some exchanges on GitHub issues as I am interested in learning how to use the learnr package for introductions into statistics via R-Tutorials. Regularly if I find a problem as a user reading and following the tutorial exercises, I try to fix the code in my fork and send the solutions as pull requests (PR) to Mine. Besides learning to use GitHub in a collaborative setting, I have already profited in many ways. I learned about packages I have never heard of (see, for example, the list of packages used for the book) and saw code snippets that transformed data in a much more elegant way as I used to do.

Summary

To inspect code on GitHub is a suitable strategy for intermediate (programming) learners. I illustrated it with an R example, but I think you could generalize it as a learning strategy. I recommend the following steps:

Note names and/or repositories you learned from successful internet recherche. By searching answers or asking questions via StackOverflow, you will encounter the same names, blog addresses, Twitter messages, repos again and again.
Look around if these person/websites generally work on problems you are interested in. Look into the profile and follow the activities of these persons on their blog, Twitter, YouTube, GitHub … accounts.
Decide on a person and select a specific project. I recommend focusing not only on a particular person but also on picking one project to concentrate on. Ideally, it is a current project the person is still interested in and produces code you want to learn and use for your own work later on.
Think about a possibility to get involved in the project. The idea is to give the person and the community something back for its/their mentoring function and commit yourself to relevant real-world problems. The possibilities depend on your skills and interests. It may be advertising the project, writing a favorable review, fixing typos, commenting via the issue part of the repo, recommending code changes via PRs, or financially supporting the author(s) and/or the ongoing project.

Finally, I would like to add also a warning: “Following” a person should not develop to stalking. It is essential not to be intrusive or pushy. You will see already after a short time if the person values your contribution(s), e.g., by answering friendly or thoughtfully, accepting your PRs, etc. If you get the impression that your engagement is disruptive, then stop it immediately. Either look at the code silently without interaction or try to find another “mentor” or project to get involved.

Using Emojis Revisited

Fri, 30 Jul 2021 00:00:00 +0000

I already reported on my learnings on the emo package in conjunction with some pull requests for interactive tutorials for the OpenIntro textbook Introduction to Modern Statistics. In the meanwhile, the book is — after some reorganization of its chapters — finished and published. I ordered the print version of the book via Amazon.

On August 13th, I started working through the tutorials and reading the new book and prepared several pull requests. There I encountered the problem of finding a special emoji without knowing the name or keyword. Meanwhile, I solved the difficulty of finding the emoji name for 🔟. There are even several solutions.

Mine Cetinkaya-Rundel referenced the EmojiTerra.com website to me. There you can search for the symbol to get all the details of the corresponding emoji.
With emo::ji_name, you will get a list of 4239 names as character strings and their corresponding value as emoji. But you have to inspect the list manually, which really is not practical. (At least I did not find a better solution programmatically.)
A more practical solution is to produce a data frame with emojis_df <- emo::jis and to inspect it via the RStudio command View(emojis_df). There you can put the emoji symbol into the search field to get the required information.
You could also get the Unicode on macOS via the window for emojis and symbols (CMD-SHIFT-SPACE) with a right-click on the icon. You will get a bunch of localized information, including the Unicode.

Screenshot of macOS window showing keycap-10 emoji

🔟
Taste mit der Zahl 10
Unicode: U+1F51F, UTF-8: F0 9F 94 9F

Summary

All in all, just options 1 and 3 are feasible in a practical manner. Either to call the EmojiTerra.com website or to use View(emo::jis) inside RStudio.

H5P and Introduction to Modern Statistics

Sun, 11 Jul 2021 00:00:00 +0000

Three week ago I received via amazon the paperback version of Introduction to Modern Statistics via Amazon. This is one of the rare exceptions where I need — in addition to the online and/or ebook version — the print edition as well. I didn’t buy the textbook just for reading but for adding educational tools, especially to this book and generally for the overall OpenIntro project.

How to integrate H5P exercises into statistics?

My idea is to generate exercises with H5P for statistics and/or data science courses applying R.

H5P is a free and open-source content collaboration framework based on JavaScript. H5P is an abbreviation for HTML5 Package and aims to make it easy for everyone to create, share and reuse interactive HTML5 content. (Wikipedia)

I am convinced that the many sophisticated but easy-to-use H5P content types are an excellent choice to support textbooks with interactive exercises. Adding H5P exercises is especially valuable for those (predominantly theoretical) textbooks in statistics that are agnostic to a particular computing language.

But my opinion is not very relevant, as I am not teaching statistics or related subjects. My conviction is motivated by my general educational expertise but not by subject knowledge on statistics or data science. As for any educational design only a cooperation between domain and didactic expert results in valuable educational interactions.

But what about books using statistical software to apply theoretical knowledge?

This question is especially relevant with R as there are some R packages designed to generate R tutorials (swirl and learnr). The motto here is: Learning statistics with R through R. In addition to these specialized packages there is also the widely used educational strategy to disseminate R Markdown files where textual guidance and R programming chunks are intermingled to form a kind of lab for statistical computing.

H5P exercises cannot replace these very practical and domain specific tutorials. At the moment I see three possible targets to add H5P exercises:

Narrative Part

Exercises for the narrative part of statistics textbooks are often sparse. In ims-book there are the formatting vehicles of examples and guided practices.

Examples

Examples will provide an understanding how methods should be applied. The textual answer (solution) follows the problem statement immediately, only separated by a line.

Guided Practices

When reader should find the solution on their own they will be presented by a Guided Practice. A footnote provides the textual answer (solution).

With online-exercises there is no place restriction. One can provide not only a multitude of exercises but also use with different media. Besides text and graphic for exercises to the narrative part also animation and video are useful educational methods. Many different types of interactions are available (click, write, drag & drop, selection, …) in different question formats (single choice, multiple choice, true/false, …).

I am also a supporter of the old but time-proven concept of multiple representations (Ainsworth 1999; van Someren et al., n.d.): The same content is learned/experienced with several distinctive exercises. This approach is not redundant as each activity addresses the brain capacity in different ways.

I developed all exercises with self-detemined respecticvely self-directed learners in mind (Ryan and Deci 2018; Deci and Ryan 2008; Knowles 1975; Mezirow 1985). Learners should solve the task to their likening. I chose the educational settings toward the highest possible freedom supported by the software (H5P content type).

Learners can choose the order of the exercises, repeat them as often as they want, and even view the solution before they start with the activity. The idea behind this strategy of maximizing educational freedom is that adult self-learners don’t want to fool themselves and are to respect in their self-organizing learning endeavor. There is one exception to the general rule of educational liberty: All activities under the sections Assessment have somewhat limited freedom so that learners could get some overall feedback about their performance.

Test: Fenced Divs

Thu, 27 May 2021 00:00:00 +0000

::: {.notices .warning} Hello world! :::

Memento Time Travel

Tue, 25 May 2021 00:00:00 +0000

Research with archived web data

In a previous article, I wrote about the possibilities of the Wayback Machine for scientific writing. I argued that archiving web pages are essential for references as they prevent link rots when cited web resources are not available anymore. With this blog entry, I am looking into the reverse option: Finding and retrieving archived web pages for research reasons.

Archived web pages as permanently stored data are indispensable for reproducibility issues. But they are also valuable research resources as they are data for historical and comparative research.

This article rewrites my blog entry from 2019 on Scraping Archived Data with the Wayback Machine in three aspects:

I am going into more detail to explain the Memento Protocol that stands behind the archiving procedures of the Internet Archive with the Wayback Machine.
I will demonstrate the research significance of archived data with another — more simple — example. Instead, to analyze yearly differences in the ranking of the popularity of static site generators, I will focus on the number of statistical packages developed by the R community over time. The website structure for static site generators is complex, has changed several times, and contradicts the demonstration purpose.¹
To argue the importance of archived data convincingly, it is necessary to extract the URLs of archived data and use these data for research issues. Therefore, I will show how R can scrape web data and display the data for further analysis.

How does the Memento API work?

Mementos are prior versions of web pages cached by web crawlers and stored in web archives. The HTTP-based Memento framework is a description for Time-Based Access to Resource States.

With Mementos, you can access a version of a Web resource as it existed at some date in the past. The complete information about the Memento Project is specified in RFC 7089 as HTTP Framework for Time-Based Access to Resource States – Memento. I will explain the essential idea with the gentle non-technical introduction on the Memento website.

In the Memento protocol there are four important components:

Original Resource (URI-R): A Web resource that exists or used to exist on the live Web for which we want to find a prior version.

Memento (URI-M): A Web resource that is a prior version of the Original Resource. Prior versions are Web resources encapsulated in what the Original Resource was like at some time in the past.

TimeGate (URI-G): A Web resource that “decides” on the basis of a given datetime, which Memento best matches what the Original Resource was like around that given datetime.

TimeMap (URI-T): A list of URIs of Mementos of the Original Resource that is archived, e.g., available online.

The central component is the TimeMap Resource. It is a machine-readable document that lists the Original Resource itself, its TimeGate, and its Mementos, as well as associated metadata such as archival DateTime for Mementos.

The HTTP-based Memento framework bridges the present and past Web. It facilitates obtaining representations of prior states of a given resource by introducing datetime negotiation and TimeMaps. Datetime negotiation is a variation on content negotiation that leverages the given resource’s URI and a user agent’s preferred datetime. TimeMaps are lists that enumerate URIs of resources that encapsulate prior states of the given resource. (Quoted from RFC 7089)

Architectural overview of how the Memento framework supports batch discovery of prior/archived versions of a resource (Mementos).

Where to find the desired information?

To demonstrate the interplay of the Memento protocol with the Wayback Machine, I will look into the history of the R project web page. I want to know the number of available packages over time.

There are three web pages where I can find out the number of available packages:

Contributed Packages

Contributed Packages: On this page is the number of available packages written directly under the subheading “Available Packages”.

Page ‘Contributed Packages’ mentions the number of available packages.

Packages By Name

Available CRAN Packages By Name: This page list all packages alphabetically. One could count the lines with the package’s name to get the desired number.

Screenshot of the web page ‘Available CRAN Packages By Name’.

Packages By Date of Publication

Available CRAN Packages By Date of Publication: This page list all packages by date of publication. Here one could also count the lines with the package’s dates to get the desired number.

Screenshot of the web page ‘Available CRAN Packages By Date of Publication’.

Which page to scrape?

The question now arises if these pages are also available in the past. And if so: Have they the same structure to use the identical CSS selector in all archived instances?

The question now arises if this page is available also in the past. And if so: Has it the same structure to use the identical CSS selector in all prior archived instances?

Contributed Packages

It turns out that the first line after the subheading could be scraped with the CSS selector #pkgs + p.

Context menue via right clicke opened to select the code inspector.

Code inspector detects CSS selector ‘h3#pgks’ and displays this result with a magnifier glass.

To check if the website structure for the paragraph after the header with id = “pkgs” remains constant, we can use the browser plugin for the Wayback Machine (available for Google Chrome and Firefox.).

Browser Plugin Wayback Machine

If you click on the Wayback Machine plugin, you can either go directly to the first or last (recent) snapshot of this web page. I choose “Overview” to display the calendar to inspect different instances of the archived page.

The calender displays time span and distribution of the archived page.

The calendar shows that the page was between May 14, 2008, and April 13, 2021, 145 times archived. From this overview page, one can select different instances and content and design of the ‘Contributed Packages’ page. To get an idea of possible changes in the structure, I start with the first instance of the archived page.

The number of times the page was crawled by the Wayback Machine (in my example: 145) has nothing to do with how often the page was updated.

Screenshot of the ‘Contributed Package’ page from May 14, 2008.

The archived page displays a different layout. The number of packages is not immediately after the first heading but after several paragraphs later (see number 2 in the image). Also, the name of the heading has changed from “Available Packages” to “Available Bundles and Packages”. But more importantly: The paragraph following immediately of this header mentions the amounts of different kinds of packages and bundles. It would be challenging to detect the desired number (in this case, 1425 packages) programmatically.

A further inspection shows that the id still remains “pkgs”. But because of the different text of the first paragraph, I want to look at another page to grab my desired information easier.

Packages By Name

Here I will demonstrate the use of a helpful tool. The Web scraping 101 vignettes of the rvest package references SelectgorGadget, a JavaScript bookmarklet to find out the required CSS selector interactively.

SelectorGadget selects the first name.

After selecting the first name, the SelectorGadget shows the CSS selector a, referring to all hyperlinks of this page. At the bottom, you can see that there are 17674 hyperlinks on this page. The problem is that the headline consists of the letters of the alphabet, which also are hyperlinks.

SelectorGadget selects the first letter of the alphabet.

To select another position on the page with the SelectorGadget, subtracts these selected items. The bottom line shows the CSS selector with ‘td a’ (table data followed by a hyperlink) and removes the 26 characters to get 17648 selected objects. So this would be an easy way to get the number of available R packages.

The first archived page starts with September 24, 2011, and reduces, therefore, our survey period. But more important: From the 193 archived pages (more than the 145 instances of the ‘Contributed Package’ page!) are many redirects.

Calender view with many green cicled dates (=redirects).

The size of the circles displays the number of pages crawled. Blue is directly archived, green is an archived page after a redirect. I could not find a solution to distinguish the difference between direct archiving and after a redirect programmatically. The problem with a redirect is that the same state of a page is archived several times but counts as different pages in time. This pollutes the resulting data set.

Packages by Date of Publication

Finally, I decided to scrape the page “Packages by Date of Publication” with the simple CSS selector ‘a’. It has the same period as the “Package By Name” page (the first instance is from September 26, 2011). It was archived only 81 times but with very few redirects.

Calendar view of the ‘Packages by Date of Publication’ page.

Memento protocol in action

Package installation

At first, we have to install and load the wayback package, a wonderful and very practical library of Memento API wrappers, written by Bob Rudis.²

In the meanwhile there exists another package with the same name on CRAN (Comprehensive R Archive Network). It is dedicated to the installation for legacy R versions and has nothing to do with the wayback package on GitHub for the Wayback Machine of the Internet Archive.

if (!require("wayback"))
        {remotes::install_github("hrbrmstr/wayback", build_vignettes = TRUE)
        library(wayback)}

## Loading required package: wayback

Mementos Link Types

The next step is to use the get_mementos() function. With get_mementos(url, timestamp = format(Sys.Date(), "%Y")) we will receive a shortlist of relevant links to the archived content. Only the first parameter, url, is mandatory. If no timestamp is provided, then the actual year is taken, and the most recent archived page will be the endpoint, which in our case is ok. The function will return the 4 link relation types as in the Request for Comment for the Memento framework described and outlined above.

Link Relation Type “original.”
Link Relation Type “timemap.”
Link Relation Type “timegate.”
Link Relation Type “memento.”

url = "https://cran.r-project.org/web/packages/available_packages_by_date.html"
cran_link_types <- wayback::get_mementos(url)
saveRDS(cran_link_types, "data/cran_link_types.rds")

cran_link_types <- readRDS("data/cran_link_types.rds")
knitr::kable(cran_link_types, 
             caption = "Memento Link Types", 
             label="memento-link-types", 
             format = "html")

Table 1: Memento Link Types
link	rel	ts
https://cran.r-project.org/web/packages/available_packages_by_date.html	original	NA
http://web.archive.org/web/timemap/link/https://cran.r-project.org/web/packages/available_packages_by_date.html	timemap	NA
http://web.archive.org/web/https://cran.r-project.org/web/packages/available_packages_by_date.html	timegate	NA
http://web.archive.org/web/20110926172444/http://cran.r-project.org:80/web/packages/available_packages_by_date.html	first memento	2011-09-26 17:24:44
http://web.archive.org/web/20210128201734/https://cran.r-project.org/web/packages/available_packages_by_date.html	prev memento	2021-01-28 20:17:34
http://web.archive.org/web/20210130222751/https://cran.r-project.org/web/packages/available_packages_by_date.html	memento	2021-01-30 22:27:51
http://web.archive.org/web/20210412212953/http://cran.r-project.org/web/packages/available_packages_by_date.html	next memento	2021-04-12 21:29:53
http://web.archive.org/web/20210412212953/http://cran.r-project.org/web/packages/available_packages_by_date.html	last memento	2021-04-12 21:29:53

Besides these 4 main types of link relations, the function also provides the first, previous, next, and last available memento. When no particular date is given, then the last memento is identical with the next (= nearest) memento. In addition to the two columns, link and rel, there is a third one, ts, containing the timestamps (empty for the first 3 link relation types). The return value in total is a tibble with eight observations (rows) and three columns.³

Memento Craw List

Providing an URL in the search field of the Wayback Machine results in the interactive browser version to the calendar view as shown by Figure 12 and Figure 13. The dates with archived content are blue or green (= redirected URL) circles. The bigger the circles, the more snapshots were archived on these dates.

We get these dated crawl lists with the get_timemap() function using the second observation of the result of the get_mementos function. This is in our case cran_link_types$link[2].

The execution of the following code chunk can take some time, depending on how many pages of the URL are archived. Be aware that the Wayback server is strained by this query, so do not repeat this operation. I store the result on my hard disk and will use the saved data for further processing.

cran_link_types <- readRDS("data/cran_link_types.rds")
cran_crawl_list <- wayback::get_timemap(cran_link_types$link[2])
saveRDS(cran_crawl_list, "data/cran_crawl_list.rds")

cran_crawl_list <- readRDS("data/cran_crawl_list.rds")
reactable::reactable(
    cran_crawl_list,
    pagination = FALSE, 
    highlight = TRUE, 
    height = 500,
    compact = TRUE,
    bordered = TRUE,
    striped = TRUE,
    wrap = FALSE,
    resizable = TRUE
    )

We get a table with 85 rows, where the first three rows are not relevant for our purpose, and the last row is empty. So we get 85 - 3 - 1 = 81 mementos, which conforms to the number of Figure 13. The URLs to the mementos are pretty long. You can widen/narrow the columns to inspect the structure of the URLs. Typically they start with “http://web.archive.org/web/” followed by the date-time string of the memento and the original URL.

Harvesting Web Pages

We have now collected all the URLs for the mementos. The next task is now getting our data from these pages and display them in a graphic. For this last step, we will use the packages rvest and ggplot2. It is a “standard” task of manipulating HTML and has only insofar with the memento framework to do, as we are using the URLs of the archived web pages.

Tidy data

We only need the memento links, date, and new column for our number of available packages.

library(tidyverse)

## ── Attaching packages ─────────────────────────────────────── tidyverse 1.3.1 ──

## ✓ ggplot2 3.3.3     ✓ purrr   0.3.4
## ✓ tibble  3.1.2     ✓ dplyr   1.0.6
## ✓ tidyr   1.1.3     ✓ stringr 1.4.0
## ✓ readr   1.4.0     ✓ forcats 0.5.1

## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## x dplyr::filter() masks stats::filter()
## x dplyr::lag()    masks stats::lag()

cran_tidy_data <- readRDS("data/cran_crawl_list.rds") %>% 
  filter(stringr::str_detect(rel, "memento")) %>% 
  mutate(date = anytime::anydate(datetime)) %>% 
  add_column(pkgs = 0) %>% 
  select(link, date, pkgs)
saveRDS(cran_tidy_data, "data/cran_tidy_data.rds")

glimpse(cran_tidy_data)

## Rows: 81
## Columns: 3
## $ link <chr> "http://web.archive.org/web/20110926172444/http://cran.r-project.…
## $ date <date> 2011-09-26, 2011-10-11, 2011-10-29, 2011-11-29, 2011-12-29, 2012…
## $ pkgs <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,…

For the purpose of this demonstration, I do not want to scrape every one of the 81 links but only one link per year. As the archiving dates do not have a systematic regularity, we cannot provide time series, say January 1st every year. But this does not matter for this demo.

cran_yearly_links <- readRDS("data/cran_tidy_data.rds") %>% 
  dplyr::mutate(year = lubridate::year(date), .after = "date")  

cran_yearly_links <- cran_yearly_links[!duplicated(cran_yearly_links[ , "year"]), ]
saveRDS(cran_yearly_links, "data/cran_yearly_links.rds")
cran_yearly_links

## # A tibble: 11 x 4
##    link                                                   date        year  pkgs
##    <chr>                                                  <date>     <dbl> <dbl>
##  1 http://web.archive.org/web/20110926172444/http://cran… 2011-09-26  2011     0
##  2 http://web.archive.org/web/20120127155954/http://cran… 2012-01-27  2012     0
##  3 http://web.archive.org/web/20130121072548/http://cran… 2013-01-21  2013     0
##  4 http://web.archive.org/web/20140402061423/http://cran… 2014-04-02  2014     0
##  5 http://web.archive.org/web/20150124092856/http://cran… 2015-01-24  2015     0
##  6 http://web.archive.org/web/20160104062049/https://cra… 2016-01-04  2016     0
##  7 http://web.archive.org/web/20170502002538/http://cran… 2017-05-02  2017     0
##  8 http://web.archive.org/web/20180822083144/https://cra… 2018-08-22  2018     0
##  9 http://web.archive.org/web/20190103070122/http://cran… 2019-01-03  2019     0
## 10 http://web.archive.org/web/20200103222850/https://cra… 2020-01-03  2020     0
## 11 http://web.archive.org/web/20210128201734/https://cra… 2021-01-28  2021     0

Get number of available packages

Although there are only 10 web pages to scrape, this will still take some time. Therefore I provide a progress indicator to monitor how much time the procedure will still last.

library(rvest)

cran_pkgs <- readRDS("data/cran_yearly_links.rds")
max = nrow(cran_pkgs)

pb <- txtProgressBar(min = 0, max = max, style = 3)
  for(i in 1:max) {
      html <- read_html(cran_pkgs$link[i])
      links <- html_nodes(html, "a")
      cran_pkgs$pkgs[i] <- length(links)
    setTxtProgressBar(pb, i)
    }
close(pb)
saveRDS(cran_pkgs, "data/cran_pkgs.rds")

(avail_pkgs <- readRDS("data/cran_pkgs.rds"))

## # A tibble: 11 x 4
##    link                                                   date        year  pkgs
##    <chr>                                                  <date>     <dbl> <dbl>
##  1 http://web.archive.org/web/20110926172444/http://cran… 2011-09-26  2011  3307
##  2 http://web.archive.org/web/20120127155954/http://cran… 2012-01-27  2012  3563
##  3 http://web.archive.org/web/20130121072548/http://cran… 2013-01-21  2013  4262
##  4 http://web.archive.org/web/20140402061423/http://cran… 2014-04-02  2014  5374
##  5 http://web.archive.org/web/20150124092856/http://cran… 2015-01-24  2015  6221
##  6 http://web.archive.org/web/20160104062049/https://cra… 2016-01-04  2016  7722
##  7 http://web.archive.org/web/20170502002538/http://cran… 2017-05-02  2017 10513
##  8 http://web.archive.org/web/20180822083144/https://cra… 2018-08-22  2018 12938
##  9 http://web.archive.org/web/20190103070122/http://cran… 2019-01-03  2019 13645
## 10 http://web.archive.org/web/20200103222850/https://cra… 2020-01-03  2020 15348
## 11 http://web.archive.org/web/20210128201734/https://cra… 2021-01-28  2021 17038

Visualizing the results

I am not going to produce a sophisticated graphic. A simple line graph to see how the number of packages is increasing has to be enough.

p <- readRDS("data/cran_pkgs.rds") %>% 
  ggplot(aes(year, pkgs, group = 1)) +
  geom_line()
p

Summary

In this post, I have presented the memento protocol, a framework to generate and retrieve prior versions of web pages cached by web crawlers and stored in web archives. I have explained how to Memento API for the Wayback Machine of the Internet Archive is working. With the wayback packages by Bob Rudis, I have demonstrated the practical handling to use the framework for retrieving, collecting, and displaying historical data.

Recently, even the URL has changed from https://www.staticgen.com/ to https://jamstack.org/generators/.↩︎
I have forked his GitHub repo and looked into his R scripts. I have to say that I became somewhat depressed as I noticed how much knowledge I am still lacking. Despite working now for four years with R, there are so many think I still have to learn!↩︎
In many cases, the last memento is identical with the memento link relation type. Then the tibble has only seven rows.↩︎

Images in Different File Formats

Fri, 14 May 2021 00:00:00 +0000

I have investigated the different methods for inserting images into various file formats and conversion procedures.

Image handling in .Rmd, .md and .Rmarkdown files

At first I looked on the various options for handling of images in .Rmd files.
Then – as a contrast program – I tested the outcomes of image handling procedures in .md files.
And finally I investigated the image procedures in.Rmarkdown files.

Observed results

The situation is complex and not easy to grasp. But on a meta-level, the case is pretty straightforward: The decision for a file format is much more important than the details in image handling.

.Rmd: The window provided via Visual R Markdown provides all its great features only in .Rmd files. With the new page bundle feature in Hugo some disadvantages of .Rmd file disappear: As every file is in its own folder the several files produced by the conversion process do not hamper anymore. But .Rmd files loose some of the nice features of the Academic theme, e.g., the table of contents on the ride side. Also the chunk option fig.align does not work with the Academic theme. To date I could not find & solve the annoying problem.
.Rmarkdown: It turns out that .Rmarkdown files have the greatest advantages. They have now almost the same features of R Markdown (writing and applying R code, using references and bibliography), but provides also the benefits of the (Academic) theme via the Goldmark renderer. Yes, you can’t use the full functionality of the Visual R Markdown window for including images but you could use instead the Hugo figure shortcode.¹

So the situation is pretty straightforward: The Hugo figure shortcode is for all different file formats a good solution. In the case of .Rmd files they have to be embedded in an HTML-chunk as demonstrated in the .Rmd post.

Additional tip

RStudio markdown snippet do not work in .md files. I do not understand why not. Ok, there is no R programming code but it coud be used for other (text) snippets. RStudio markdown snippet do work with .Rmarkdown files. So I have prepared an snippet to include the Hugo figure shortcode syntax and saved under ‘ff’. So I just need to type ff + SHIFT-Tab to get a template with all the parameters .

At the moment I do not know if using the figure shortcode results in some processing penalties. At least I read and experienced it myself that with many images the standard time-out parameter for Netlify has to be set much higher than the standard 10 seconds. ↩︎

Images in Rmarkdown Files

Fri, 07 May 2021 00:00:00 +0000

Using R chunk with `knitr`

Using R chunks with knitr has the full features as outlined in the post Images: From R Markdown to HTML format.

Example

Figure 1: Caption for this figure 1

Code in R Chunk

```
{r img-with-knitr, echo=FALSE, fig.align='center', out.width='100%', fig.cap='Caption for this figure 1'}
knitr::include_graphics("images/my-image.png")
```

Code in HTML

```
<div class="figure" style="text-align: center">
    <span id="fig:img-with-knitr"></span>
    <img src="images/my-image.png" alt="Caption for this figure 1" width="100%"/>
    <p class="caption">Figure 1: Caption for this figure 1</p>
</div>

```

CSS

div.figure {
        border: 1px;
        border-style: groove;
        box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2), 0 6px 20px  rgba(0, 0, 0, 0.19);
}

p.caption {
    text-align: center;
    margin-top: -0.5rem;
    margin-bottom: 0.5rem;
    font-size: smaller;
}

Summary

Caption: yes
Format of caption: no
Caption automatically numbered: yes
Alt: yes
Title: no
Tooltip: no
Width/Height: yes
Link to: no
ID: yes (format: “fig:<chunk name>)
Classes: yes (with out.extra, example: out.extra='class="border shadowed"'
CSS style: yes (with out.extra, example: out.extra='style="background-color: #9ecff7; padding:10px; display: inline-block;"' See blog post Tips and tricks for working with images and figures in R Markdown documents
Other (key=value): no

Markdown via Addins ‘Insert Image’

The behavior is the same as in Images in .md files.

Using the RStudio Addin window without the width or height parameter it just generate markdown code. Is the width or height parameter included then HTML is generated but without div.figure and p.caption class. There is no caption visible and the CSS styling has to be done with <img> tag.

Example

You can’t use the Addins ‘Insert Image’ in Visual R Markdown mode because it protects the conversion by adding a backslash in front of the squared brackets: !\[Alt text\](images/my-image.png).

Code in markdown

The first line is the markdown code for images without width/height parameters. The second line has a width parameter.

![Alt text](images/my-image.png)
<img src="images/my-image.png" alt="Alt text" width="100%"/>

Code in HTML

The first line is the HTML code for images without width/height parameters. The second line has a width parameter.

<p><img src="images/my-image.png" alt="Alt text"></p> 
<img src="images/my-image.png" alt="Alt text" width="100%">

CSS

If the RStudio Addin window is used, then the img tag should be wrapped into a paragraph. Otherwise it would be generate a conflict with other tags related with images.

p img {
    border: 1px;
    border-style: groove;
    box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2), 0 6px 20px  rgba(0, 0, 0, 0.19);
}

Summary

So – all in all – the Addins window is not very useful. The only advantage is the generation of the filepath and the copy of the image in the correct folder.

Caption: yes
Format of caption: yes
Caption automatically numbered: no
Alt: no
Title: no
Tooltip: no
Width/Height: yes
Link to: no
ID: no
Classes: no
CSS style: no
Other (key=value): no

Again the same behavior as in Images in .md files.

With a .Rmarkdown file Visual R Markdown offers only a reduced first part of the window as it is presented in an .Rmd file. In contrast to the .Rmd version it lacks the fields to add width and height parameter.

But more important: There is no caption! The text for the field ‘Caption/Alt’ includes the text only for the alt-attribute!

Example

Code in markdown

![Alt text (no caption!)](images/my-image.png "Text for title and tooltip")

Code in HTML

<p><img src="images/my-image.png" alt="Alt text (no caption!)" title="Text for title and tooltip"></p>

CSS

The same as with the RStudio Addin window.

Summary

Caption: no
Format of caption: no
Caption automatically numbered: no
Alt: no
Title: yes
Tooltip: yes
Width/Height: no
Link to: yes
ID: no
Classes: no
CSS style: no
Other (key=value): no

Hugo figure shortcut

Here again – as in the .md file – we do not need to protect the shortcode. We can insert it either via the Visual R Markdown menu Insert -> Shortcode or write the code directly into markdown.

Example

Figure 4: my title-text

Everything I said about the features of the figure shortcode in the post Images: From R Markdown to HTML format applies for .md and for .Rmarkdown files as well.

Summary

Caption: yes
Format of caption: no
Caption automatically numbered: yes (Academic theme)
Alt: yes
Title: no
Tooltip: no (zoomable)
Width/Height: yes
Link to: yes
ID: yes (Academic theme)
Classes: yes
CSS style: yes
Other (key=value): no

Results and Conclusion

Attribute	knitr	Addin	Visual	Hugo	Remark
Caption	✅	❌	❌	✅
C. format	❌ 1	❌	❌	❌ 1	1) standard = bold
C.numbered	✅	❌	❌	✅ 2	2) via ‘Academic’ theme
Alt	✅	❌	❌	✅
Title	❌	❌	✅	❌
Tooltip	❌	❌	✅	❌
Width/Height	✅	✅	❌	✅
Link to	❌	❌	✅	✅
ID	✅ 3	❌	❌	✅ 4	3) `fig:<chunk name>` 4) `figure-<caption text>` or ‘Academic’ theme
CSS	✅	❌	❌	✅
Other	❌5	❌	❌	✅ 6	1) via output chunk arbitrary HTML code! 4) `target`, `rel`, `attr`, `attrlink`

Table: Features of different method of inserting images in .Rmarkdown files converting to .markdown using Pandoc and Hugo/Goldmark.

Although both windows (Addins and Visual R Markdown windows) are not equipped with all the necessary attributes, here we have in contrast to .md files also the alternative with knitr chunks. So in way: .Rmarkdown files are the best of both worlds! (This is also true with the Table of Contents, which works here as well.)

Images in .md Files

Thu, 06 May 2021 00:00:00 +0000

Markdown via Addins ‘Insert Image’

Example

Code in markdown

The first line is the markdown code for images without width/height parameters. The second line has a width parameter.

![Alt text](images/my-image.png)
<img src="images/my-image.png" alt="Alt text" width="100%"/>

Code in HTML

The first line is the HTML code for images without width/height parameters. The second line has a width parameter.

<p><img src="images/my-image.png" alt="Alt text"></p> 
<img src="images/my-image.png" alt="Alt text" width="100%">

CSS

If the RStudio Addin window is used, then the img tag should be wrapped into a paragraph. Otherwise it would be generate a conflict with other tags related with images.

p img {
    border: 1px;
    border-style: groove;
    box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2), 0 6px 20px  rgba(0, 0, 0, 0.19);
}

Summary

So – all in all – the Addins window is not very useful. The only advantage is the generation of the filepath and the copy of the image in the correct folder.

Caption: yes
Format of caption: yes
Caption automatically numbered: no
Alt: no
Title: no
Tooltip: no
Width/Height: yes
Link to: no
ID: no
Classes: no
CSS style: no
Other (key=value): no

With a .md file Visual R Markdown offers only a reduced first part of the window as it is presented in an .Rmd file. In contrast to the .Rmd version it lacks the fields to add width and height parameter.

But more important: There is no caption! The text for the field ‘Caption/Alt’ includes the text only for the alt-attribute!

Example

Code in markdown

![Alt text (no caption!)](images/my-image.png "Text for title and tooltip")

Code in HTML

<p><img src="images/my-image.png" alt="Alt text (no caption!)" title="Text for title and tooltip"></p>

CSS

The same as with the RStudio Addin window.

Summary

Caption: no
Format of caption: no
Caption automatically numbered: no
Alt: no
Title: yes
Tooltip: yes
Width/Height: no
Link to: yes
ID: no
Classes: no
CSS style: no
Other (key=value): no

Hugo figure shortcut

In contrast to the .Rmd file we do not need to protect the shortcode. We can insert it either via the Visual R Markdown menu Insert -> Shortcode or write the code directly into markdown.

Example

Figure 4: my title-text

Everything I said about the features of the figure shortcode in the post Images: From R Markdown to HTML format applies for .md files as well.

Summary

Caption: yes
Format of caption: no
Caption automatically numbered: yes (Academic theme)
Alt: yes
Title: no
Tooltip: no (zoomable)
Width/Height: yes
Link to: yes
ID: yes (Academic theme)
Classes: yes
CSS style: yes
Other (key=value): no

Results and Conclusion

Attribute	Addin	Visual	Hugo	Remark
Caption	❌	❌	✅
C. format	❌	❌	❌ 1	1) standard = bold
C.numbered	❌	❌	✅ 2	2) via ‘Academic’ theme
Alt	❌	❌	✅
Title	❌	✅	❌
Tooltip	❌	✅	❌
Width/Height	✅	❌	✅
Link to	❌	✅	✅
ID	❌	❌	✅ 3	3) `figure-<caption text>` or ‘Academic’ theme
CSS	❌	❌	✅
Other	❌	❌	✅ 4	4) `target`, `rel`, `attr`, `attrlink`

Table: Features of different method of inserting images in .md files converting with Hugo/Goldmark to HTML

Only Hugo shortcode (with ID and numbered extension by the Academic theme) is the only real choice for figures equipped with all necessary attributes and nice styled.

Images: From R Markdown to HTML format

Wed, 05 May 2021 00:00:00 +0000

Introduction

This article is the first of a series of posts to investigate methods of inserting figures into different markdown formats. The resulting code depends on how images are included and the conversion tool they have to undergo. And — at least I think — also somewhat on the theme in use. In my case it is the blog widget of the starter-academic theme.

Here I look at the process chain from Rmd via Pandoc to HTML. I will inspect inserting images with the following methods:

R chunk code via knitr::include_graphics().
‘Insert Image’ window via blogdown Addins
‘Image …’ window vial Visual R Markdown
Hugo shortcode

Of course, you can always include HTML instead of markdown code, but this case is not of interest here as there are no changes during the conversion process.

Using r chunk with `knitr`

You can include external images with the knitr function include_graphics(). There are many chunk options to specify the resulting figure, but not all are useful for external images.

Use out.width and out.height (not fig.width or fig.height, they are for R-generated plots)
Use out.extra to add CSS styles (see for details Style your image environment with CSS)
If fig.cap is provided without fig.alt, it will also function as alternative text, which is usually not desired.
Use output hooks if you want even more control.

Example

Figure 1: Caption for figure 1

The fig.align parameter generally works correctly, but not in this theme. It always centers to a 10 (instead 0f 12) column width. Left and right alignments do not work at all. I could not find and solve the problem.

Code in R Chunk

```
{r img-with-knitr, echo=FALSE, fig.align='center', out.width='50%', fig.cap='Caption for figure 1', fig.alt='Alternative text for figure 1'}
knitr::include_graphics("images/my-image.png")
```

Code in HTML

```
<div class="figure" style="text-align: center"><span id="fig:img-with-knitr"></span>
    <img src="images/my-image.png" alt="Alternative text for figure 1" width="50%">
    <p class="caption">
        Figure 1: Caption for figure 1
    </p>
</div>
```

It is a stand-alone feature of knitr to provide automatically generated figure numbers that you can reference via their id. The id is generated from the chunk name with the pattern fig:<chunk name>. The figure 1 above can be referenced with \@ref(fig:img-with-knitr). Like this:

For more details see Figure 1.

But in that case, you cannot use other methods to insert figures with numbers as knitr will not be aware of them.

The relevant selectors for CSS styles are div.figure and p.caption. We will see that these styles are consistent with all the other methods provided by RStudio windows. But Hugo’s figure shortcode uses the HTML tags figure and figcaption. In the R Markdown Coobook is a work around expained how you can produce these Hugo HTML tags with knitr.

CSS

The CSS style I have used to display the image:

img {
        border: 1px;
        border-style: groove;
        box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2), 0 6px 20px  rgba(0, 0, 0, 0.19);
}

p.caption {
    text-align: center;
    margin-top: -0.5rem;
    margin-bottom: 0.5rem;
    font-size: smaller;
}

Summary

Using the knitr function include_graphics() for external images provides a rich set of image attributes (tags). It is also very flexible, especially if you are using the more complex possibilities like programming the output chunk hook.

Caption: yes
Format of caption: no
Caption automatically numbered: yes
Alt: yes
Title: no
Tooltip: no
Width/Height: yes
Link to: no
ID: yes (format: "fig:<chunk name>)
Classes: yes (with out.extra, example: out.extra='class="border shadowed"'
CSS style: yes (with out.extra, example: out.extra='style="background-color: #9ecff7; padding:10px; display: inline-block;"' See blog post Tips and tricks for working with images and figures in R Markdown documents
Other (key=value): no

Markdown via Addins ‘Insert Image’

The possibilities of RStudio Addins window for blogdown are sparse. But it has one big advantage: The addin helps user to provide the correct file path and copies the image to the right place.

The RStudio Addins window for blogdown package allows only width / height arguments and alternative text. The content of the ‘alt’-attribute is in the final HTML code converted to the caption, which is not correct:

The alternate text should describe what you can see on the image in case the image does not appear (e.g., is blocked) or the user applies a screen reader. ‘Alt’ text is also rendered by search engines. Every picture should have a corresponding ‘alt’ attribute The caption by contrast is not so important. It provides additional information and appears under the image explaining the content to reader who are able to see the picture.

Example

Figure 2: Caption for this figure 2

You can’t use the Addins ‘Insert Image’ in Visual R Markdown mode because it protects the conversion by adding a backslash in front of the squared brackets: ![Alt text](images/my-image.png).

Code in markdown

RStudio Addins window produces the following markdown code:

![**Figure 2:** Caption for this figure 2](images/my-image.png){width=100%}

I think starting with the RStudio version 1.4 (January 2021) the RStudio Addin window is superseded by the the Visual R Markdown window (see next section). If you are turning on the visual editor you can always enhance the rudimentary markdown of the addin window with all the features of Visual R Markdown.

For this improvement you need to

turn on Visual RStudio Markdown;
select the image which markdown you want to enhance;
click into the small round button with the three points to open up the dialog window;
correct or add the desired content for the attributes (there is more under the “Attributes” tab);
close the window;

If you want to work with markdown directly you may turn off Visual RStudio Markdown.

Screenshot of the Visual RStudio Markdown window

Code in HTML

Here I am referring to the RStudio Addin without enhancement from the Visual R Markdown window.

<div class="figure">
    <img src="images/my-image.png" alt="">
    <p class="caption"><strong>Figure 2:</strong> Caption for this figure 1           
    </p>
</div>

As you can see: The alt attribute is empty, although I have filled it out!

If you do not provide text for the alt (= capture) attributes then there is no div.figure and p.caption class. In this case there is only the img tag as you can see with the image under the header Markdown via Addins ‘Insert Image’.

CSS

The conversion to HTML produced exactly the same classes as in the result with the knitr image inclusion (see the CSS code under the knitr section).

Summary

Caption: yes
Format of caption: yes
Caption automatically numbered: no
Alt: no
Title: no
Tooltip: no
Width/Height: yes
Link to: no
ID: no
Classes: no
CSS style: no
Other (key=value): no

Visual R Markdown via Menu ‘Insert …’

After you have turned on Visual R Markdown you can load the image window via the Menu 'Insert ' -> 'Image…' or shortcut ⇧⌘I. For an overview I put in the picture above both parts of the window side by side. You can see that it offers a very complete list of image attributes.

If you do not include text in the “Caption/Alt” field then no HTML code for the div.figure and p.caption classes will be produced and the appropriate CSS styles cannot be applied.

Example

Figure 3: Caption for this figure

Even in this enhanced window is the wrong mixture of ‘Caption’ and ‘Alt’ attributes offered. But there is a work around with the ‘Other’-field as I have shown in the screenshot.

Fill in the field “Caption/Alt” and add the alt attribute in the field “Others”. After the conversion to HTML the former constitutes the caption and the later the alternative text.

If you click on the inserted image and call the window again, then a slightly different window appears where fields for width and height are added and can be changed. This addition appears under the image as well by just clicking on the image.

Width and height field added after the image was included into the .Rmd page

Code in markdown

![Figure 3: Caption for this figure](images/my-image.png "title text and tooltip"){#figure3 alt="here comes a description for blind people, screen readers and search engines}

In addition to the standard markdown code all the other attributes are added inside curly brackets.

Code in HTML

<div class="figure">
    <img src="images/my-image.png" title="title text and tooltip" id="figure3"
    alt="here comes a description for blind people, screen readers and search engines">
    <p class="caption">
        Figure 3: Caption for this figure
    </p>
</div>

CSS

The conversion to HTML produced exactly the same classes as in the result with the other two methods knitr image inclusion and Addins ‘Insert Image’).

Summary

Caption: yes
Format of caption: no
Caption automatically numbered: no
Alt: no
Title: yes
Tooltip: yes
Width/Height: yes
Link to: yes
ID: yes
Classes: yes
CSS style: yes
Other (key=value): yes

Hugo figure shortcut

You cannot simply write {{< shortcode >}} in R Markdown, because Pandoc is not aware of Hugo shortcodes, and may convert special characters so that Hugo can no longer recognize the shortcodes (e.g. < will be converted to <).¹

With Blackfriday as default Markdown rendering engine I used to use for shortcodes the blogdown::shortcode() function. But after Hugo v0.60 the default Markdown rendering engine changed to Goldmark. In the above quoted man page for the shortcode() function there is a also special note on this issue:

Since Hugo v0.60, Hugo has switched its default Markdown rendering engine to Goldmark. One consequence is that shortcodes may fail to render. You may enable the unsafe option in the configuration file: https://gohugo.io/getting-started/configuration-markup/#goldmark.

I couldn’t understand this remark as for me all the Hugo shortcodes works well. But to apply it more easily I use pure HTML code chunks to prevent a conversion by Pandoc. This has the advantage that I can include Hugo’s shortcode syntax without any changes.

Example

My caption text

Code in HTML chunk

```{=html}
{{< figure src="images/my-image.png" class="center" alt="My alt text" caption="My title text" id="shortcut-figure" numbered="true" >}}
```

The id and numbered attributes are not provided by but they are a feature from the academic theme.

If you do not use id explicitly then Hugo generates an ID with the “figure-” followed by the caption text (e.g., id="figure-my-caption-text").
You can see that the numbering systems starts from scratch, e.g., it does not account for the knitr chunks. The same is true vice versa.
Besides the already mentioned attributes Hugo also features target, rel and attribution (attr = text; attrlink = link).

CSS

```{=html}
<figure id="figure-shortcut-figure">
  <div class="d-flex justify-content-center">
    <div class="w-100">
        <img alt="My alt text" srcset="
               /post/images-from-rmd-to-html-format/images/my-image_huf81a7faf920bb11d47d82b059b9dc0cf_20503_1061e20606536f4413fb34665581822b.png 400w,
               /post/images-from-rmd-to-html-format/images/my-image_huf81a7faf920bb11d47d82b059b9dc0cf_20503_c4c18b47561549b1147ccf6dd152b5f0.png 760w,
               /post/images-from-rmd-to-html-format/images/my-image_huf81a7faf920bb11d47d82b059b9dc0cf_20503_1200x1200_fit_lanczos_2.png 1200w" src="/post/images-from-rmd-to-html-format/images/my-image_huf81a7faf920bb11d47d82b059b9dc0cf_20503_1061e20606536f4413fb34665581822b.png" width="754" height="226" loading="lazy" data-zoomable="" class="medium-zoom-image"></div>
  </div>
  <figcaption data-pre="Figure&nbsp;" data-post=":&nbsp;" class="numbered">
      My caption text
  </figcaption>
</figure>

```

The produced HTML is pretty complex, but here we are just interested in the two relevant tags for CSS styling: <figure> and <figcaption>. These two HTML tag are different to the code produced by other methods of inserting images. So maybe it would be a good idea to style both variants the same way.

Summary

Caption: yes
Format of caption: no
Caption automatically numbered: yes (Academic theme)
Alt: yes
Title: no
Tooltip: no (zoomable)
Width/Height: yes
Link to: yes
ID: yes (Academic theme)
Classes: yes
CSS style: yes
Other (key=value): no

Results and Conclusion

Table: Features of different method of inserting images in Rmd files converting to HTML
Attribute	knitr	Addin	Visual	Hugo	Remark
Caption	✅	✅	✅	✅
C. format	❌ ²	✅	❌	❌ ²	² standard = bold
C.numbered	✅	❌	❌	✅ ³	³ via ‘Academic’ theme
Alt	✅	❌	✅ ⁴	✅	⁴ via ‘Other’-field
Title	❌	❌	✅	❌
Tooltip	❌	❌	✅	❌
Width/Height	✅	✅	✅	✅
Link to	❌	❌	✅	✅
ID	✅ ⁵	❌	✅	✅ ⁶	⁵ `fig:<chunk name>` ⁶ `figure-<caption text>` or ‘Academic’ theme
CSS	✅	❌	✅	✅
Other	❌¹	❌	✅ ⁷	✅ ⁸	¹via output chunk arbitrary HTML code! ⁷ free choice! ⁸ `target`, `rel`, `attr`, `attrlink`

If we take into account that code produced by the RStudio Addin window can be improved at any time with the Visual R Markdown window then all the four offered methods are pretty similar. But keep in mind that advantages on the one hand are often counter acted by disadvantages on other areas. So — for instance — provides the conversion via Pandoc many advantages but looses also some features of Hugo or the applied theme.

To sum up this this little investigation I see three patterns of decisions criteria:

One big difference is the possibility to number the figures automatically. The Addin and Visual R Markdown windows do not provide this feature.
On the other hand filling in the window fields is — especially for people who are not so comfortable with HTML or Markdown code — easier.
Another decision criteria could be consistency: If you are using R chunks (which is the major reason to use the .Rmd format!) and want to be consistent throughout the file, then the decision for the knitr method is obvious.

But let’s see what the other file format are producing. For people using different methods² for their posts perhaps the comparison inside one file format is not sufficient. Perhaps there is a method not ideal when considered only one file format but has a higher consistency when looked all file format together.

Helper functions to write Hugo shortcodes using the R syntax↩︎
As with blogdown version 1.3 not all files are automatically rendered, one could choose a different file and conversion format for every post separately. This requires (1) deleting those files produced by the conversion process in the folder of the post under consideration (2) renaming the file ending of the main file and kniting/rendering the new file format again.↩︎

Using Emojis in R and R Markdown

Mon, 19 Apr 2021 00:00:00 +0000

Recently I worked on some pull requests for interactive tutorials for the OpenIntro textbook Introduction to Modern Statistics. There I came across the package emo. Its purpose is: “Easily insert emoji into and RMarkdown”.

I was surprised that there is a special package. I didn’t think that including emojis wouldn’t be a problem – at least on the macOS. I only have to open the window for emojis and symbols (CMD-SHIFT-SPACE) and double click on the icon to insert it into the markdown document. No big deal, so what’s the matter?

Screenshot of macOS window showing a selection of emojis

After I searched online, I saw that there many questions (and solutions) about this problem. There is on CRAN a special package emojifont for using emojis and Font Awesome in base R and ggplot2 graphics and with emoGG another one on GitHub.

For me, it is still not clear what exactly the problem is. Perhaps it has to do with different operating systems? Does the Unicode not translate to all systems?

Anyway, I looked into emo, and it turned out that it offers some useful functions when working with strings containing emojis. You can also add emojis to summarize p-values. Additionally, I learned several things:

With emojis_df <- emo::jis, you get a data frame with 3570 emoticons categorized by 21 variables.
In emojis_df, you will find the keywords for each emoji. But this is problematic as they are not unique. In the ReadMe of emo in GitHub, you will see a massive list of multiple references for each keyword.
If you are using keywords, the command emo::ji(<keyword>) will randomly pick one.

emo::ji("face")
emo::ji("face")
emo::ji("face")

## 😈
## 💆 
## 👼

To identify, you have to use the short name of the Common Local Data Repository (CDLR) included in the emojis_df. If you want to use emojis with a particular skin tone, you have to use the alias name as the CDLR name produces an error. Instead of looking up an emoji within the above-created data frame emojis_df, I found it more practicable to search in the official Unicode Emoji List. It has not so many different columns but focuses on icon display and CLDR short name.

How to Register with a Hosting Service to Buy Space for your Server

Wed, 16 Dec 2020 00:00:00 +0000

With this article, I begin a new series of step-by-step instructions focused on installing and working with open source tools on your own server space. I’ve learned some of these procedures participating as an audience member in a sequence of German web talks by Guido Brombrach, organized by OERcamp.

Step 1: Go to the URL of your hosting service

Step 1: Go to the URL of Uberspace and scroll to the middle of the start page.

Go to the start page of your hosting service. This tutorial uses Uberspace.
Scroll to the middle of the page and click on the link SIGN ME UP!

Step 2: Provide your username

Step 2: Choose username and fill it in the form of the Uberspace registration page.

Think again about your chosen username. You can’t change it anymore. It will be the first part of the URL, followed by the server address.
Fill in your username and tick the check box to conform to the technical and social house rules.
Check if your username is still available.

Step 3: Provide your email address (to reset your password)

Step 3: After creating your account Uberspace asks for additional information. First part: eMail.

Uberspace comes back after some moment with this new form. Meanwhile, the program has stored your username in the Uberspace database and already logged you in. What you here see is the first part of the form.

Also, you have already gotten an email address.
But for security reasons, I recommend providing another email address. Whenever you forgot your password, you can apply for a new password sent to your email account. Otherwise, you are locked out from your webspace.

Step 4: Set your password and choose your price model

Step 4: Uberspace asking for additional information. Second part: Password and price.

After scrolling down, you will see this second part of the form.

Provide a password to enter later the admin pages of your webspace.
This password has nothing to do with the password you will use to access the server via SSH or SFTP. (I will explain Secure Shell resp. Secure File Transfer Protocol in another tutorial.)
Uberspace has an exciting price model. The first month is free in any case. After that, you have three paying options. All of them have the same services but differ in your financial contribution to the hosting service. You can always later change your decision.
Then click the “Make it so” button.
Be patient. It can take several minutes till your server space is ready for you to access.

Step 5: Success of web registration!

Step 5: Success page of the web registration process.

When you see this page, your registration process was successful. Congrats!
There are many helpful resources in the Uberspace manual and wiki. But if you are still stuck, don’t be shy and ask the people at Uberspace directly via email.

Step 6: Password for SSH access

Step 6: Uberspace start page after logged in.

The successful registration for web access only means that you can now enter your uberspace web admin page. To get the page you see on the screenshot, you have to login into Uberspace and open the tab “Logins.”
On the left side, you see your login for the Uberspace admin page (your web access). If you want, you can provide another password that will override your old one. As you can see, under 2a, I am already logged in.
What we need to do now is to provide a password for access to your server space. When you fill in this (different) password, you will get a red bar on top of the screen saying you have set an SSH password. With this password, you can now access your server with an FTP-client or via your terminal.

Educational Tools and Learning Theory

Sun, 12 Jan 2020 00:00:00 +0000

This article suggests a procedure to categorize the enormous amount of educational tools in the market. My approach uses a framework of three well-grounded learning theories: Instruction, Cognition, and Construction based on presentation modes (one-way teaching), dialog settings (two-way education), and collaborative scenarios (learning by mastering complex situations). As a proof of concept, I will use the Top Tools for Learning (TT4L), compiled by Jane Hart as a result of the 13th Annual Learning Tools Survey (published 18 September 2019).

Inspired by the yearly ranking of Top Tools for Learning (TT4L) compiled by Jane Hart, I wanted to know if there are global changes or trends in e-learning over the years. The ranked list of tools goes back 13 years and is available for all past years via the Wayback Machine. As a plan for analysis and for presenting results, I imagine a similar procedure as I have used in comparing static website generators over the years.

Inconsistencies in Jane’s categories

To compare the different rankings of the educational tools, I will need a classification of all apps/web services based on pedagogical reasons. For my purposes, the categorizations used by Jane Hart (see right column) is not useful:

It is the functionality of the tool itself, which motivates her division in different classes primarily. Therefore all categories contain the word ‘tool’ in their name. She distinguishes four main types:
1. Instructional tools
2. Content development tools
3. Social tools
4. Personal (and professional) tools
Her sub-categorization under the four main headings is not consistent and has changed over the years. Sometimes she uses ‘Content’ as a fifth type, or she includes ‘professional’ under the heading of ‘personal tool.’
Some tools are platform-dependent (e.g., Sharepoint, Keynote).
Even proprietary name for hardware is ranked in her list (e.g., iPhone, iPad, SMART board instead of smartphone, tablet, whiteboard).

Another much better division can be drawn from her beautiful infographics (see below) on top tools for higher education 2019. Here Hart distinguishes by pedagogical reasons between

Didactics (Formal Learning)
Discovery (Informal Learning)
Discourse (Social Learning) and
Doing (Studying).

Aside from the fact that ‘formal’ and ‘informal learning’ has another meaning (see the definition by the Council of Europe on formal, non-formal, and informal learning). I do not understand why ‘studying’ is different from structured learning in an academic setting. And why should ‘studying’ be linked just with the doing-bubble?

Usage of top tools for higher edudation in 2019. Infographics by C4LPT (2019): https://www.toptools4learning.com/edu100/.

Developing educational motivated categories

Three different learning/teaching modes. Infographics by Peter Baumgartner (2018)

Without going here into detail, I will use my categorization of learning theories published among others in The Zen Art of Teaching (PDF, 545 kB). This article outlines three different kinds of teaching modes:

Instruction or Learning I: Using static learning materials for reading, listening, and watching.
Cognition or Learning II: Using dynamic and elaborated didactical educational interactions for guided practice.
Construction or Learning III: Using collaborative scenarios for mastering unstructured situations.

This categorization results in different educational aspects for each of the three teaching/learning modes:

Kind of knowledge acquisition: static –dynamic/procedural – social.
Teaching modes: transfer – dialog/discussion – collaboration.
Learning strategies: study – practice – mastering.
Assessment methods: correct answers – applying adequate techniques, processes, and procedures – success criteria.
Teaching strategies: explaining – observing and helping – collaborating.

Even though my suggestion is coarse-grained as it distinguishes only three types, it has to overcome several problems of operationalizations:

One tool – several functionalities

Sometimes it is not easy to classify tools unambiguously into one of the three categories. Modern tools incorporate many features that may address different types of learning modes. Take, for example, the use of a learning management system (LMS). You can use it for various activities (reading PDFs, work through quizzes, or writing collaboratively into a wiki). In those inconclusive situations, I will utilize the highest predominant functionality of the tools. ‘Highest’ means that Learning II is more elevated than Learning I. This is not a quality rating of the different learning modes as all three modes have their value under certain conditions. ‘Higher’ or ‘lower’ address here the complexity of student activities.

But what is the ‘predominant functionality’ of a tool? I assume that every educational technology, every technology-supported learning environment, or internet application, implement a theoretical learning model – irrespective of opinions and beliefs of developers and teachers. In the case of LMS dominates guided learning interactivity (Learning II). Although teachers may use tools ingeniously for purposes not foreseen from the developers, I think that these inventing teaching strategies are not the mainstream.

Following other users on the web with the same subject interests, getting in contact with them, commenting their blogs, etc. entitles for Learning II but does not qualify for Learning III. I will rate only rate an educational setting as Learning III if learners produce (individually or collaboratively) an artifact for learning purposes.

General tools not intended primarily for education

In Jane Hart’s list of top tools are many products and web services not primarily developed for education, like word processors, email clients, file-sharing platforms, note-taking applications, browsers, search engines, etc. The reason for this broad mixture is the content of the questionnaire and their target group. Not learners or teachers, but mainly IT professionals and learning designers, addresses the survey. This orientation leads to two biases, as mentioned by Jane Hart:

only 22% of voters came from colleges and universities, hence the Top 200 list is therefore skewed towards workplace learning.
since only 6% of respondents came from schools, the EDU100 list this year focuses on tools for Higher Education.

Snippet of survey form for compiling the top tool list for 2020

Another issue favoring general tools are the questions themselves. See, for instance, the voting form for 2020 on the right ahnd side of this article:

A valid entry needs to include 10 different DIGITAL tools (resources, services, etc) – listed in any order (emphasis not mine).

So the questions ask for digital tools, not for digital educational tools. The consequence is that the top list with a few exceptions does not include subject-oriented software (e.g., apps for learning math, science, languages). This bias is an essential drawback as many e-learning scenarios use these kinds of content-related software.

The other side of the same coin is that there is a bias towards personal knowledge management applications. Software like email clients, browsers, search engines, etc., are unquestionably also essential parts of a learning ecology. Using these tools always goes together with (informal) learning experiences. But an educational taxonomy of tools for education should focus on apps or web services developed for e-learning. To distinguish those personal productivity instruments from tools explicitly dedicated to learning, I have designated them with Learning 0. Under this category, I also have classified support tools like converters or ‘neutral’ web resources like portals for icons, images, or other material.

Summarizing this subsection is the following types of tools designated with Learning 0:

General productivity: Word processors, email clients, browsers, search engines, converters, translators, grammar checkers, image/icon/theme libraries, and other supporting tools.
Personal knowledge management: Note-taking software, calendars, project management, skill validation.
Hardware: Smartphones, tablets, interactive whiteboards.

Developing learning material or using it?

Another complication arises from the fact that the educational judgment of developer tools can be seen two-fold:

from the authoring perspective, e.g., the resource development view
from the angle of using the developed product

There are content development tools which can be used by learner and teacher alike. Those apps or web services with various perspectives (e.g., blog services, screen-casting, curation tools) I will classify from their ownership. If it is easily possible that students get their personal instance, then I will designate it as Learning III.

With ‘easy’ I mean

that it is free available at least in a freemium variant for personal usage. This definition of ‘free available’ does not apply if learners are entitled to free usage only via an institutional account.
that it is easy to install either through the download of packaged software without any special additional technical requirements, such as using the terminal or running a server, changing functionality programmatically, or only through a simple online registration).
that it can be used easily in educational settings (no specialized knowledge necessary, learn time only 1-2hrs, no specialized hardware, platform-independent).

Note that every case has to be judged individually. For a correct classification with the mentioned rules, one has to look into pricing models, installation procedures, user interface, and typical application patterns.

Conclusion

To summarize my suggestion for an educational categorization of digital tools, I will define four types of apps, respectively web services:

Learning 0: Tools supporting personal knowledge management, productivity, and portals for web resources.
Learning I: Transfer of knowledge. The focus of learner engagement is primarily oriented to content to acquire factual knowledge.
Learning II: Applying knowledge, practicing procedures. The emphasis here is learner interactivity with dynamic content and dialog with teacher and co-learner.
Learning III: Collaboratively solving challenges. Essential is producing a useful output, doing something creatively and cooperatively.
Learning IV: Multi-purpose tool with features for Learning III and either Learning 0 or Learning I.

… to be continued

Update 2021-05-20: Originally, I have planned to reorganize Jane Hart’s data according to my set of categories. But unfortunately, because of other priorities, I didn’t have time so far. But the follow-up of this post is still on my schedule.

DORA and Wakelet: Two Badges!

Wed, 25 Dec 2019 00:00:00 +0000

Wakelet Community Badge

Last week – just before Christmas – I received two badges: DORA (San Francisco Declaration on Research Assessment) sent me a newsletter about their advances for fostering Open Science in 2019. On this occasion, DORA also issued badges for individuals and organizations that have signed the declaration.

The other badge came from Wakelet. As I had applied to be part of the Wakelet community, they welcomed me with an email with some tips and hints and issued the Community Badge to me.

DORA Signatory Badge

So just in time for Christmas Eve, I am honored with these two gratifications. I thought about a little thank-you gift. What I came up with was a Wakelet collection of DORA’s newsletter, which essentially is a commented list link. I turned these links into a ‘Wake’ with some added value: Besides the DORA list, there is additional information in every linked section generated automatically by Wakelet. Additionally, the link list is now more visually appealing. So I killed two birds with one stone: A Wakelet about DORA’s advances of the year 2019! Enjoy! (There is more on my Wakelet Homepage.)

Wakelet - Share Visually-Engaging Stories

Mon, 16 Dec 2019 00:00:00 +0000

What is a Wakelet?

Several months ago, I came across Wakelet. My first impression was that it is another social bookmarker because you can “bookmark anything you find online in two clicks.” But after some days working with it, I noticed it is more than that.

The reason for my changed awareness of Wakelet is its feature to create stories. Arranging bookmarks into “collections,” you can organize your bookmarks, adding text, and include images to get your narrative across. With Wakelet, you curate and share content not only in a meaningful but also in a visually-engaging way.

How do People use Wakelet?

I think the best way is to illustrate the possibilities and benefits of Wakelet is by showcasing some excellent examples. I have grouped my collected Wakelets, and to demonstrate the range of different options, I have always provided two instances for each category.

This is the first and prominent use of content curation, especially for education. @realUcyJoy is as an educational entrepreneur and advocate passionate about the use of education technology in Africa to improve learning. She shares a collection of 20 Digital skills for educators with us. Another very sophisticated design of a Wakelet resource page has @drbexl (Dr. Bex Lewis) created.

Portfolio

My name twin @FelixBaumgartner (I am not related to him) famous for his spectacular skydive uses Wakelet to promote himself and his adventurous projects. All collections on his homepage serve as material for media publications. Another showcasing page maintains Beth Tweddle @bethtweddlenews), a gymnastics Olympic medallist.

Raising awareness

The Rohingya Crisis @Rohingya uses Wakelet for political reasons to draw support to stop the genocide, ethnic cleansing and numerous crimes against the Rohingya. The Glasgow Centre for Population Health @theGCPH documents its conferences on providing leadership for action to tackle inequality and improve health via a series of tweeds, as can be seen by the example of its Glasgow’s Healthier Future Forum 20.

Philippe Schuler @Phil demonstrates his love for sports, drums, photography, nature, and travel with some astonishing photographs. Another example of this kind of usages gives Teijas ky @teijasky, one of the developers of Wakelet. He binds his amazing marine biology photos from the Galapagos Islands into a story of the history of the geological and biological processes of this archipelago.

Education

There are many examples of educational applications of Wakelet. Educational scenarios with Wakelet are so crucial for the developers that they dedicated a particular homepage for learning communities. There is a specialized guide for educators, translated into several languages providing tips and tricks. (Surprisingly, a guide in German is still missing.) Some example Wakes are Ancient Greek Philosophy for learning and The EdTech Toolbox - 2019 for teaching. See the blog of Kathleen Morris and Online Tools for teaching and learning for more learning activities and educational scenarios with Wakelets.

Figure 1: Six Use Cases for Wakelets

Wakelet and the SAMR model

Ruben Puentedura’s SAMR model exemplifies the meaningful integration of technology into teaching. Instead of using “tech for tech’s sake,” the SAMR model features a view inspired by social constructivism.¹

Figure 2: Getting started with Wakelet: Graphic by Kathleen Morris

Substitution: Students read an article on Wakelet instead of in class.
Augmentation: Wakelet allows for multimodality by enabling users to embed images, videos, articles, and more.
Modification: Students curate information and resources about a specific topic on Wakelet and can share it widely via social media for feedback. Students critically analyze content and research to create an archive for discussion.
Redefinition: Students have access to real-time collaboration on multimodal content curation.

Wakelet works seamlessly with various apps

Figure 3: Ten different options to include content into a Wake.(Screenshot)

You can embed “Wakes” (= collection of items) into your web pages as I have done with the three referenced Wakelets above. (You can’t produce Wakes from Wakelet homepages.) There are three different formats to choose from and some design options how to present the Wakes.

The example above is an embedded Wake referring to other Wakes. Click on the different sections to get the referenced Wakes. You can also produce PDFs from your Wakes, but the PDFs are not interactive anymore. Look at this PDF example.

Storing content is just a two click action with its browser extensions, available for Firefox, Chrome, and Safari. Wakelet comes with mobile apps for Android (Google Play) and iOS (App Store) and it works also seamlessly with various tools like

Microsoft Teams
Microsoft OneNote — See: Better together: OneNote and Wakelet.
Google Drive — See: Google Drive Integration
Google Classroom
Remind
Fliprid — See: Flipgrid Integration

Wakelet is free and always will be

Wakelet also has a refined help system featuring many short but informative videos explaining the different highlights. Wakelet also comes with a specialized guide for educators in several languages. (Surprisingly, a guide in German is still missing.)

Furthermore, Wakelet is confidence-inspiring as it has sensible rules of usage and stringent privacy policy as well. And the best of it:

Wakelet is free and always will be. All of the features available now will remain free forever, even if we introduce premium plans in the future.

Feature requests

There are a two features I wish Wakelet would include. Both have to do with collaboration and sharing. I am not sure if my desires are in line with the developer’s strategy and plans. But anyway, here they are:

Genuine collaboration: At the moment, I can only invite contributors to add links but not to change or to delete the work I have done. So there is no real interworking in the sense of cross-functional cooperation possible. All the participants work in parallel but not as coordinated team players.
Community evaluation: Sharing links is okay but lacks quality control. I would like to have feedback from the community either as comments or as attached discussion fora. Tools like stars to appraise Wakes, as well as links, would also foster the community discussion and showcase the best collections. Make this a feature to turn on/turn off, so that people could decide if they want to draw critiques and spend time for discussions.

Update 2021-05-19: Both my feature requests are in the meanwhile realized.

Collaboration: The creator of a collection can offer a QR code, an alphanumeric code, or an URL. You can join a collection by one of these three options. You do not even have to log in! Watch this video. — Another (even newer) feature is “spaces”: With just several clicks, users can create a “space” to house content to be shared.

Community evaluation: Several icons (like 👍, 👏, 💯) are added so that visitors can give their “likes” to the collection in general or to individual links. Admittedly this is only a rudimentary evaluation, but you always can open up your wakelet for free collaboration so that other people can add additional links or comments.

Conclusion

Wakelet is not only a free sophisticated tool for content curation but also well-suited for visually-engaging presentations and story-telling. It has a steadily growing user base. Already some of the world’s most prominent organizations are using Wakelet.

Figure 4: Prominent organizations using Wakelet.

Wakelet encourages both the sharing of resources and their joint creation and curation of material. Still, in this area, some more features for cooperation would be desirable. A strict privacy policy combined with a well-engineered support page offering many short videos and helpful articles and a responsible team creates confidence for a long future of this product. And even if there comes a time where other similar products gain prominence, you can save your work via Wakelet’s export feature (JSON and PDF format) and change the web service presumably without too many troubles.

Figure 5: The Wakelet team.

I follow with these text passages the above-quoted “Online Tools for teaching and learning” ↩︎

Announcing the R package coinsR

Fri, 02 Aug 2019 00:00:00 +0000

Update 2021-05-21: Because of some breaking changes in the Hugo site structure and management in the last three years, coinsR works at the moment only with individual pages.

This blog entry announces the R program coinsR to produce bibliographic metadata automatically for websites within the Hugo framework. The dominant use case at the moment is with the blogdown package.

In a previous blog entry I wrote about the advantages of providing bibliographic metadata for your web pages producing COinS. COinS is the abbreviation of Context Objects in Spans. It refers to a microformat standard to embed bibliographic metadata as HTML.

A COinS string includes HTML code for all the information necessary to cite a publication correctly. It works for every type of writing (books, papers, web pages). COinS embedded in your HTML pages help your blog readers or website visitors to cite your posts and pages correctly.

I wrote also a detailed tuorial how to produce COinS with the help of the free bibliographical research tool Zotero. But this is a quite complicated procedure with several steps. Besides, it works only for every page individually and not automatically for all blog entries of your website.

To produce COinS for every web page quickly was my motivation to write the R package coinsR.

Use cases of `coinsR` and its limitations

The package requires:

Hugo with the R package blogdown
Config file in the TOML format (config.toml)
Blog entries with YAML frontmatter

I plan for a future version to loosen the last two limitations and allow config.yaml and TOML frontmatter (but not for .json).

Update 2021-05-21: This plan is still relevant: I would like to loosen some restrictions and improve the user interface.

You can call coinsR via an RStudio Addin for every page individually. But you can also produce COinS automatically for all files under a specific directory.

Where to find `coinsR` and how to install it?

You can download the package from Github with

remotes::install_github('petzi53/coinsR', build_vignettes = TRUE)

I am trying to prepare coinsR as an R package distributed via the CRAN repository. But as I am still not very experienced in R programming, I am not sure if I will succeed.

You will find a detailed description in the help file and the vignette description of how to invoke the two functions coins_this() and coins_all().

Advantages of URL shortener

Tue, 23 Jul 2019 00:00:00 +0000

Citing and visiting web addresses with long URLs is sometimes complicated. Long URLs are particularly a hardness when there are no clickable links but only long strings printed on paper. This article suggests URL shortener to avoid the hurdle mentioned above.

What is a URL Shortener?¹

At first, we need to clarify what an “URL” is. “URL” stands for “Uniform Resource Locator,” and they’re a way of identifying the location of a file on the internet.

A URL is not the same as a domain name! Sometimes these both terms are confused and used interchangeably. But a domain name is just one part of a URL. The image below shows the difference:

Figure 1: Parts of an Uniform Resource Locator (URL)²

Use URL shortener for your references

Depending on the Content Management System (CMS) the website is using you can get very long URLs. There are three types of long URLs:

Long pretty URLs: These URLs consist of understandable text strings. Example: https://www.dbfk.de/media/docs/expertengruppen/pflegemanagement/DBfK-Handreichung-BAG-Pflegemanagement-Anwerbung-auslaendischer-Mitarbeiter_innen.pdf
Long ugly URLs: These URLs include randomly mixed characters, generated automatically by the CMS. Example: https://www.art.com/gallery/id--c23945/decorative-art-prints.htm?ui=6731D67F76A44083AD1B8F110C4FE301.
Long mixed URLs: These URLs contain understandable parts, but these sequences are directed to the CMS and not to humans. Examples are strings for the search engine: https://www.sfgate.com/search/?action=search&firstRequest=1&searchindex=solr&query=%22earthquake+san+francisco%22.

The solution for all these kinds of long URLs is the usage of URL shortener. There is a great variety of URL shortener available. But instead to use the automatic generated short URL (e.g., something like bit.ly/2GIYPa3) use the possibility to edit the URL with an appropriate and memorable word. Although these strings are often longer than the automatically generated short URL, they are much easier to type. Besides, you or other users can even memorize them (e.g., https://bit.ly/url-shortener).

As other users of the service also take memorizable short URLs, sometimes there may be a clash of URLs. If this happens, then you are not allowed to generate this specific URL. To prevent colliding URLs, I use a short prefix referring to me as the author or to the subject/project of the URL (e.g., https://bit.ly/pb-url-shortener).

Summary

You might be thinking that shortening your URL is an extra step, but URL shorteners are very beneficial:

Short URLs are more comfortable to share
Short URLs are essential if there is limited space (e.g., Twitter)
Short URLs are more visually appealing
Short URLs are memorized easier
Short URLs are more straightforward to type when there is only a paper reference
Short URL services can provide additional information such as the number of clicks
Short URLs can be adjusted if the original URL changes, so that users still have the correct links
Short URLs services allow you sometimes to create your brand and categories for your URLs. If they do not provide this service, you can use prefixes to get some rudimentary management for your URLs
Short URLs prevent line breakings (e.g., in emails) so that the user does not have to cut and paste to put the link together

But there is also a disadvantage of URL shortener:² They are often used by spammers to hide the URL destination. That way, unsuspecting people are more likely to click on them.

Wakelet

In addition to the following Wakelet, there is also a community edition on my Wakelet homepage where you can add relevant links.

I took the next two paragraphs from Oberlo and modified them slightly. ↩︎
I took the next two sentences from Oberlo. ↩︎

Archiving Quoted Web Resources

Mon, 22 Jul 2019 00:00:00 +0000

Quoting web resources is a hassle for several reasons:

Web pages are not available anymore.
Web pages have moved to another URL.
Web pages change their content so that the cited reference is not correct anymore.

Humanities, where detailed content analysis of websites is a popular research method. Referring to exact quotes is a question of reproducibility and therefore crucial in science generally. This article presents some strategies and tools to bypass the challenges mentioned above.

Quote websites with the Wayback Machine

With WebCite, there used to be a web service to circumvent link rot and changed the content. WebCite allowed to archive online resources and returned an URL where these filed pages could be accessed. Besides that this service was often down and therefore notoriously unreliable, as of July 14, 2019, it does not accept any new archive requests anymore.

Figure 1: Start page of the WebCite service, proclaiming that new archiving request are currently not feasible anymore.

Luckily with Wayback Machine¹, operated by the Internet Archive, there is recently a new and reliable web service available. Although there is a sophisticated how-to use of this service in the Wikipedia context, I have prepared my own How-to use Wayback Machine for the general public.

Visit my slides on How-to use Wayback Machine for the general public.

Figure 2: Start page of the Wayback Machine, a service by the Internet Archive

How to cite archived resources?

Internet Archive asked the Modern Language Association (MLA) how to cite resources archived with the Wayback Machine. MLA Style is a prevalent system for documenting sources in scholarly writing.

MLA answered

that there is no established format for resources like the Wayback Machine, but it’s best to err on the side of more information. You should cite the webpage as you would normally, and then give the Wayback Machine information.

MLA also provided an example:

McDonald, R. C. “Basic Canary Care.” Robirda Online. 12 Sept. 2004. 18 Dec. 2006 $$<http://www.robirda.com/cancare.html>$$. Internet Archive. $$ http://web.archive.org/web/20041009202820/http://www.robirda.com/cancare.html$$.

Note there are several additions to a standard bibliography:

Two dates: The first is the date of the archive, then comes the date when the page is retrieved.
Two URLs: The first is the original URL (not available anymore), then comes the archived URL from the Internet Archive.
Web service: Between the two URLs comes the ‘second’ author, the name of the internet service which archived the resource and generated its URL.

According to MLA, both URLs shouldn’t be underlined in the bibliography.

Let’s try another example. The archiving service Peeep.Us is not available anymore. The Wayback Machine gives us as archived URL https://web.archive.org/web/20180813205348/http://peeep.us:80/. If we are are going to compose this bibliography in the usual way, we would get:

Nikolaev, Cyril. “Peeep.Us.” Save Snapshot of a Web Page Forever!, 13 Aug. 2018, https://web.archive.org/web/20180813205348/http://peeep.us:80/.

Using a name for web sites may questionable, but I use it whenever there is a reasonable possibility (e.g., from the Copyright or from the name of the institution, which produces the web site).

Now we have to add the retrieval date, the original URL and the name of the archiving service:

Nikolaev, Cyril. “Peeep.Us.” Save Snapshot of a Web Page Forever!, 13 Aug. 2018, 22 Jul. 2019 $$<http://peeep.us>$$. Internet Archive. $$<https://web.archive.org/web/20180813205348/http://peeep.us:80/>$$.

Wakelet

In addition to the following Wakelet, there is also a community edition on my Wakelet homepage where you can add relevant links.

To access this page you must be registered by archive.org. ↩︎

Open Citations - TOS

Tue, 25 Jun 2019 00:00:00 +0000

Introduction

This is the second post of a series of ten contributions about a better understanding of the different aspects of Open Science. In this post, I will outline the rationale and significance behind the Open Citation movement, to collect material for the development of a taxonomy of Open Science (TOS).

The following graphics summarizes my proposal for the first level of a taxonomy of Open Science (TOS). Branches with red pointers are active links connecting to the corresponding posts I have written so far.

Figure 1: The first level of a suggested Taxonomy of Open Science (TOS)

Citations: An essential activity during the research process

In contrast to Open Access, the movement on Open Citation is not so well known. However, to build a full-fledged ecosphere for Open Science, it is essential that citations are freely available, downloadable, machine-readable, and reusable.

The following two quotes explain the significance of Open Citations:

While the act of citation by the author may be the work of a moment, the citation itself, once the citing work is published, becomes an enduring component of the academic ecosystem. Open Citation Definition

Citations are the links that knit together our scientific and cultural knowledge. They are primary data that provide both provenance and an explanation for how we know facts. They allow us to attribute and credit scientific contributions, and they enable the evaluation of research and its impacts. In sum, citations are the most important vehicle for the discovery, dissemination, and evaluation of all scholarly knowledge. I4OC: Initiative for Open Citations

Even if an article is published as Open Access, its citations are not automatically Open Citations. To qualify as Open Citations, the publisher must fulfill some conditions:

Three obligatory requirements for Open Citations

Citations must be structured in a way that they can be accessed programmatically.
Citations must be accessed separable from their sources, such as journals articles or books.
Citations must not only be free accessible but also reusable.

The origins of this movement can be traced back to OpenCitaton, a project funded by JISC, a UK based organization, which provides digital solutions for the education and research. In 2016 the Initiative for Open Citation was launched, which today is the driving force behind the movement. It aims for free availability and usage of all metadata from publications with a digital object identifier (DOI) registered by Crossref. Freely available citation data are accessible through the Crossref program interface or the Open Citation Corpus. Open Citations can be used to find publications, but also for the analysis of the citation corpus as well (e.g., “how do different fields of knowledge fit together?”).

Figure 2: How many citations referenced by Crossref are Open Citations? (see: https://web.archive.org/web/20190623212728/https://i4oc.org/, accessed 2019-06-23.)

Keep in mind that the figures above refer only to those citations referenced by Crossref. The relation of all scientific publication to Open Citations is much worse. The biggest problem is that Open Citations are not in the business interests of two key players: Elsevier (Scopus) and Clarivate Analytics (Web of Science).

Web of Science (WoS):

WoS (previously known as Web of Knowledge) is a commercial online citation indexing service owned by Clarivate Analytics.

Clarivate Analytics was formerly the Intellectual Property and Science division of Thomson Reuters. In 2016 Thomson Reuters struck a 3.55 billion dollar deal in which they spun it off into an independent company and sold it. Wikipedia. See more in detail the The Scientist, a magazine for life science professionals.

Figure 3: Clarivate Analytics Web of Science product portfolio

WoS is not only one product but platform with many different indexing services and several scientific literature search databases. The central product in the portfolio of Clarivate Analytics is the Web of Science Core Collection (see Figure 3). It includes:

Science Citation Index (SCI),
Social Sciences Citation Index (SSCI) and
Arts & Humanities Citation Index (A&HCI).

From the WoS Core Collection data set derives the Journal Impact Factor (JIF), published in the yearly Journal Citation Reports (JCR). Although the JIF is seriously flawed, almost all academic institution e require their researcher to play by the rules of the JIF. Only in the last few years, the critiques gather speed. So signed to date (2019-06-24) already 1,415 organizations and 14,467 individual researchers the San Francisco Declaration On Research Assessment (DORA) against the Journal Impact Factor. But not only critique but also the development of bibliometric alternatives (altmetrics) gain importance. I will cover bibliometric measures and these recent developments in other posts more in detail.

WoS has a tremendous impact on the behavior of researchers and their career development. To date (2019-06-24) Clarivate Analytics covers the following numbers of academic publictions¹

Category	WoS Core	WoS Platform
# of journals	> 20,900	> 34,200
# of records	> 73 million	> 155 million
Cited references	1.4 billion	1.6 billion

To put these figures into perspectives: They cover “only” between 35% (Natural Sciences) to 12% (Arts and humanities) of all journals (> 62,500) as listed in UlrichsWeb (Ulrich’s Global Serials Directory) [@mongeon_journal_2016],

Clarivate also possesses other vital tools and services for scholarly research:

Endnote, a popular reference management software, formerly the property of Thomson Reuters.
Publons: is a service for academics to honor respectively showcase their scientific work which does not lead to a “standard” publication in one of the JIF journals. The name of the enterprise is an homage to the moniker publon, signifying the smallest publishable unit. This concept is a cynical reference to the phenomenon that for the academic career the number of publications is often more important than their individual quality, resulting in “salami slicing” of papers.
Kopernio is a technology startup, which developed a web-browser extension that simplifies the process of finding and legally downloading scholarly publications.

Clarivate Analytics acquired Publons in 2017 and Kopernio in April 2018. These purchases in recent years demonstrate that Clarivate Analytics knows how to secure its leading market position: Both services are (currently) free and are doubtless useful for the individual researcher. Besides generating revenues from publishers, Clarivate binds academics to their main product as both free services are closely related and linked to WoS.

This kind of effective use of the academic community is a well-known strategy, illustrated by two more examples.

First example:

In September 2008 Thomson Reuter – at that time still the owner of Endnote – started a lawsuit with the argument of copyright infringement for US$10 million against the developer of Zotero, the Center for History and New Media at the George Mason University’s. Back then, I wrote about this lawsuit in my German blog (Endnote klagt Zotero auf 10 Mio $ and Zotero sieht der Klage gelassen entgegen).

Thomson Reuters criticized that Zotero has reverse engineered their Endnote bibliographic citation styles where each style addresses the particular requirement of a journal. Reuter saw it as a violation of the site license agreement, especially as Zotero transformed these bibliographic styles into the XML-based open Citation Style Language.

However, the crux of the matter is that members of the community developed all these Endnote styles hosted on the Endnote website. Imagine a situation where Microsoft Word claims to be the owner of all MS Word templates, designed by us, the users! It seems that this is the way for many endeavors in academia: We scientist do the whole work free (e.g., peer review) and the commercial enterprises sell it (e.g., as quality assurance for their journals). – BTW: Thomson Reuters lost the case against Zotero (Wikipedia)

Second example:

As helpful the services of Pablons for the scientists are, we have to keep in mind, that academic work by the research community on the Clarivate website are freebies in exchange to just higher visibility of their research. Researcher track their publications, citation metrics, peer reviews, and journal editing work not only for free but it is hardly any surprise, that their writings are imported from Clarivates WoS, their Endnote bibliographic reference manager (bought with 250 US$ from Clarivate) and their citation metrics come from the Web of Science Core Collection, owned again by Clarivate.

Scopus (Elsevier)

Elsevier has even more market power than Clarivate Analytics. It owns

Scopus, a database of abstracts and citations with a coverage from about 47% (Biomedical journals) to 18% (Arts & Humanities)
Science Direct, a database for scientific publications and ebooks (inclusive medical journals), which sells via subscriptions
Mendeley, a desktop and web program for bibliographic management but also a social networking website for academics with to date (2019-06-24) more than 6 million users

In contrast to WoS and Google (with Google Scholar another big player in the citation reference business): Elsevier does not only sell the usage of their database but is also the owner of a vast list of journals themselves. From the perspective of this double ownership, Elsevier’s business model is a closed circle: It includes the paid use of their database so that academics can find and cite scientific literature. The result of these citations is an increase of reputation of Elsevier’s journals through a higher Journal Impact Factor.

Elsevier is infamous for his incredibly high-profit margin, which is about 35-40%. In contrast, financial institutes and banks work with 10-15%, and the much-criticized Walmart has only about 3% profit. These figures come from the excellent and free accessible documentary “Paywall: The Business of Scholarship” [@schmitt_paywall_2018]. In other posts, I will dwell more about the business model and the nasty role of Elsevier in the Open Science movement.

However, back to the Open Citations issue regarding Elsevier: It turns out that Elsevier is the biggest obstacle for a better proportion of Open Citations:

… of all 956,050,193 references from journal articles stored at Crossref, 305,956,704 (32.00%) are from journal articles published by Elsevier, none of which are in the Crossref “Open” category, freely available for others to use.
Put another way, of the 470,008,522 references from journal articles stored at Crossref that are not open, 305,956,704 (65.10%) are from journals published by Elsevier (Open Citations Blog).

Summary

In this post, I have outlined the rationale and significance of the Open Citation movement. Citations reflect the structure and relationship of our scientific and cultural knowledge and deserve research in its own right. As “dwarfs standing on the shoulders of giants” (Isaac Newton and nowadays also the motto of Google), we generate our knowledge from previous discoveries. Citations are the expression of a social network of interconnected links which itself are due to scientific research. Much can be learned of this interplay of different researchers, subject areas, and language communities through different times and regions.

I have also argued that the economic interests of two key players in the research business are obstacles to overcome for a higher rate of Open Citations. Clarivate Analytics form together with Elsevier a duopoly and maybe with Google even a tripoly [@schoolworkhelper_business_2018]: Because of the competition between two or three sellers they cannot work like a monopoly and dictate without consideration their market condition. However, they can work in a kind competition-cooperation relationship; an economic framework called the coopetition paradox [@raza-ullah_coopetition_2014].

Add relevant links to the subject of Open Citation on my Wakelet page.

References

The category “journals” includes books, conference proceedings, and data sets. ↩︎

Toward a Taxonomy of Open Science (TOS)

Mon, 24 Jun 2019 00:00:00 +0000

This post starts a series of ten contributions about a better understanding of the different aspects of Open Science. I want to collect material to develop a taxonomy of Open Science (TOS). The primary goal of this undertaking is not only to build a hierarchical system where every notion is unambiguous but to develop a heuristic tool useful for further research.

The many faces of Open Science

With a superficial approach to the topic, it almost looks as if Open Science is identical to Open Access. Indeed, Open Access represents within Open Science the strongest and far-reaching movement with the highest financial and political consequences. However, Open Access is only one (important) part of the Open Science movement, which consists of a wide variety of different viewpoints, each with distinct socio-political conditions and effects. The planned series of post on Open Science will show how extensive and diverse this area already is. A first impression about the complexity of the subject communicates the following graphic [@pontika_fostering_2015].

Although I consider this compilation of the numerous fields of Open Science to be valuable, I see – sensitized by my own work on taxonomies [@baumgartner_feedback-arten_2016; @baumgartner_taxonomie_2014-5; @baumgartner_potential_2009] – in Figure 1 some inconsistency and shortcomings:

I miss some critical areas such as Open Content, Open Educational Resources, and Open Licenses.
There are theoretical discrepancies and overlaps, e.g., Open Science Workflows is under Open Reproducible Research. In my opinion we need different workflows not only in the area of reproducible research but in all Open Science activities. Another example: Irreproducible Research, is classified as a sub-item of Open Reproducible Research. However, how is it possible that the contraction of a thing is a sub-category of the very same thing? Irreproducible Research is not a part of Open Reproducible Research; it is plain and simple a misnomer.

Figure 1: Open Science Taxonomy: Originally published in (Pontika et al. 2015, 3). See also the interactive graphic, where all terms are linked to additional material on the Foster website (FOSTER consortium 2015).

Maybe the above figure was not really intended to be inherently consistent? As there is another version with live links to different subject worked out by the members of the project team, the diagram maybe just functions as a starting point to explore in a more systematic way the various relevant topics? Anyway: I believe that a self-consistent taxonomy would a helpful for a better understanding and holistic perspective of Open Science.

I can’t here already provide a comprehensive alternative proposal for a taxonomy. For a logically consistent counterdraft of a taxonomy I would need to look more detailed into all the different aspects of Open Science. At them moment my understanding and knowledge on Open Science is still elementary.

But what I will do here, is to suggest a different first level for a taxonomy of Open Science (TOS). Hopefully this will work out as a more precise and coherent starting point for further research. My recommendation corresponds to the nine elements of the first level of the FOSTER taxonomy: Open Access, Open Data, Open Reproducible Research, Open Science Definition, Open Science Evaluation, Open Science Guidelines, Open Science Policies, Open Science Projects, and Open Science Tools.

My proposal contains also nine elements for the first level, but they differ essentially form the FOSTER taxonomy as the following graph shows:

Figure 02: Toward a taxonomy for Open Science (TOS)

I believe that the essence of Open Science is a particular (new) set of activities during the research process. These nine categories designate a particular set of activities.To describe these activities in detail is the keypart of a better understanding about Open Science. I have already stated in another post, that the “Open” in Open Science refers to the participatory way of knowledge creation and the shared usage of its products. Therefore we are going to look into the different phases of the research process and its exploitation.

In the next couple of weeks, I will therefore look into these activities of the research process. Hopefully this will provide me nd a deeper understaning what kinds of processes Open Science constitute. This should provide me with the material for the third tier of a taxonomy for Open Science.

Every time I will publish one of these articles, I will repeat my graph with a slight change: You will see a new red arrow to the left of the relevant first level category. Behind these marked categories you will find active links to the corresponding post.

References

FOSTER consortium. 2015. “Resources | FOSTER.” The Future of Science Is Open. https://www.fosteropenscience.eu/resources.

Pontika, Nancy, Petr Knoth, Matteo Cancellieri, and Samuel Pearce. 2015. “Fostering Open Science to Research Using a Taxonomy and an eLearning Portal.” In Proceedings of the 15th International Conference on Knowledge Technologies and Data-Driven Business, 11:1–8. I-KNOW ’15. New York, NY, USA: ACM. https://doi.org/10.1145/2809563.2809571.

Bibliographic Metadata for your web page

Wed, 19 Jun 2019 00:00:00 +0000

With Web 2.0, we see a radical change in scholarly communication. On the one hand, there is a growing need for researchers to be present on different web channels (blog, twitter, youtube, and much more). On the other hand, the more traditional ways of publications in high ranked peer review channels are still prevalent. It is difficult for researchers to meet this twofold challenge at the same time: After all the day has only 24 hours!

Double bind for ECRs

Figure 1: EURAXESS - Researchers in Motion is a unique pan-European initiative delivering information and support services to professional researchers.

Especially for young researchers (so-called Early Career Researchers, or ECRs¹ is this situation alarming and even existence-threatening. The find themselves in a double-bind: Should they more invest in traditional ways of career planning, or is it more promising to expand and foster their internet presence? I do not have a clear solution for myself. I think a transition period, the most secure strategy is to follow both paths.

ECRs have both the most to gain and the most to lose from being at the forefront of changes to scholarly communications [@eve_open_2014; @st._louis_open_2015].

COinS and Zotero

Figure 2: Zotero is a free, easy-to-use tool to help you collect, organize, cite, and share research.

The only way I can think of to meet both challenges is to maximize the efficiency of all career moves. One of my aims with this blog on Open Science Education is to talk about technology supported new ways for research and publication to fulfill both requirements.

With the following lines, I will show you how you can help your blog readers or website visitors to cite your posts and pages correctly and in way that they could count as a kind of publication.

My suggestion is

to use the free research tool Zotero
to produce a specific HTML code (COinS = Context Objects in Spans) and
inject this code into your web page.

COinS is a microformat standard to embed bibliographic metadata for Embedding in HTML. COinS include as HTML code all the information necessary to cite a publication correctly. It works for every type of publication (books, papers, web pages).

Figure 3: Zotero is designed to store, manage, and cite bibliographic references, such as books and articles. It is available for Mac, Windows, and Linux.

There are several possibilities to generate COinS, but I will focus on Zotero. I am not going into further details here, but I have prepared a tutorial with images from all the necessary steps to produce, embed, and download COinS.

References

There are different definitions for ECRs around, varying how many years after a Ph.D. are included. The period starts from 4 years and goes to in some institutions to 10 years. A more detailed description of different research profile descriptors has the European Research Commission (EURAXESS) ↩︎

How to produce bibliographic metadata for your web page?

Wed, 19 Jun 2019 00:00:00 +0000

In an accompanying post I explain why to provide bibliographic metadata for your web entries could be beneficial.

Requirements

In this tutorial, I will show how to produce and embed bibliographic metadata into your web page. To follow this guide, I assume that:

you have installed the standalone version of Zotero.
you have installed the appropriate Zotero connector (plugin) for your browser.
you have a working knowledge how to use Zotero (There are many (video) instructions on the internet, see for instance the ‘official’ Zotero Quick Start Guide or the introduction by the Idaho State University).

Bibliographic metadata in 14 small steps

Start Zotero and your browser. Go to the page where you want to embed the metadata. You see the Zotero icon for web pages in the place where your web browsers shows the symbols for its installed extensions.
Figure 01: Target web page for embedding bibliographic metadata.
Click on the icon and download the rudimentary bibliographic information for the web page.
Figure 02: Download the rudimentary bibliographic information with Zotero.
Change to Zotero and inspect the new downloaded entry. You will notice that many details are still missing.
Figure 03: Zotero bilbiograpy with the new downloaed biobliographic entry.
Change the item type to an appropriate value: in my case “Blog Post”. You will see that the Zotero icon changes to reflect the new value. Different item types have different fields.
Figure 04: The standard web page entry misses many information in its bilbiographic record.
Add the relevant information as shown on the screenshot. Depending on the chosen item type, there are different fields to fill in. There is a list of item types, fields, and their meanings on a Zotero help page.
Figure 05: Adding information into the bibliographic record.
When you bibliographic entry is complete you can start the conversion and export procedure. Open the context menu (on MacOS is it a right mouse click).
Figure 06: Opening the context menu of the bibliography entry to start the export.
From the appearing drop down menu choose the “COinS” translator.
Figure 07: Choose COinS as export translator.
Save the converted data in a file on your hard disk. The place is not essential as we only need the text inside the file. After we have copied the COinS data into our web site, we will delete our intermediary file.
Figure 08: Prepare to save the converted information.
File name and file extension are also irrelevant. Just check if your export format is “COinS”.
Figure 09: MacOS Save File dialog window with export mode “COinS”.
Open the exported file with your text editor and select the COinS HTML code. You will see the text of your bibliography record URL encoded in the COinS standard format. URL Encoding replaces unsafe ASCII characters with a “%” followed by two hexadecimal digits. But don’t worry, it is not necessary to understand the code snippet.
Figure 10: Exported COinS code as it is seen after opening the file with a text editor.
Copy and paste the code in the source code of your web page. Again, the place does not matter. On a life web page visitor do not see these code but only the result – the particular Zotero icon.
Figure 11: Embed the exported COinS code into the HTML source code of the web page.
Now inspect you life web page. You will see that the Zotero icon has changed from “Web Page” to “Block Post”. Congratulation you made it!
Figure 12: Inspection the modified web page.
We need to check if everything works as intended. Save the new generated bibliographic metadata into your zotero database.
Figure 13: Download of the new generated metadata to test if everything works as intended.
Check if all the bibliographic metadata are imported correctly.
Figure 14: Inspection of the modified bibliographic record in Zotero

Wrapping up and additional information

You have learned to create, embed, and download bibliographic metadata from those web pages where you have access to the source code. There are further options in addition to what I have demonstrated in the tutorial:

You are not limited to the item type “Block Post”: You can choose from all the 34 different types of bibliographic entries. But watch out: Every item type has its (slightly) different input form and Zotero icon. You need to know what kind of information each bibliographic record type needs.
You are not limited to one COinS entry: You can generate and embed as many bibliographic entries you want. If your web page has a list of references, you can offer visitors of your web page to download the bibliographic information for the full bibliography. The Zotero icon changes to a folder if there is more than one bibliographic entry to download.
You do not need to generate a COinS entry by yourself if the bibliographic entry has a DOI: I do not know how this works, but Zotero recognizes DOI entries. See as an example, my blog post about data science education.

Data Science Education with coursera

Tue, 18 Jun 2019 00:00:00 +0000

Beginning in December 2016, I initiated a new personal enterprise: Learning the statistical programming language R to acquire competencies of a data scientist. With “learning enterprise,” I do not mean to get just interested in a new subject, to read from time to time a relevant book and to look into some web-based tutorials. No, with “enterprise” I mean a much more significant undertaking, namely to focus and concentrate several months on a systematic study for a new set of qualifications.

I started with a nine-part (June 2019: now ten-part) course specialization to data science, taught by three professors of John Hopkins University (Jeff Leek, Roger D. Peng, and Brian Caffo) and offered via the MOOC¹-platform Coursera.

One-Sided Course Philosophy

But after I have finished successfully two courses with certificates, I turned impatient with the teaching style and course philosophy. My dissatisfaction did not so much arise by these three professors but had more abstract reasons: The course has as target audience people who need certified knowledge to get into (a better) job position. People like me (experienced adult learner, who are already permanently employed and have no intention to change the job) who want to learn specific skills for their working needs, are not well supported. I do not need to provide evidence of some acquired general qualifications as most of the course participants do. I was instead looking for some Personal Knowledge [@polanyi_personal_1974-2], which I could combine with my life and working experiences. After a long time again in the learner role, I found it very strange that I had to spend much learning time on material arranged by other people which I didn’t need and had, on the other hand, to learn some stuff superficially or to skip altogether, which is essential for my use cases.

To exemplify the problem in more detail: In the third course on Getting and Cleaning Data we had to learn different ways to get data into the R environment: To download it with URLs, to scrap it from web pages, to load it from Excel sheets, to collect it via an API. But – for instance – we didn’t learn how to get data from an SQL database. I appreciate that all these different methods are conceivably relevant, and as a becoming data scientist, one should know how to apply them all. I also understand that it is impossible to learn all the different use cases in every detail. But I disagree that the best teaching method is an exemplary journey through some of these potentially essential methods!

A teaching strategy which would have better suited me and other self-determinded learners would have been a systematic and complete list of all approaches with their pointers to the relevant R packages and supplemented with some prototypical program snippets to get started. After a systematic overview and instructional material of all techniques, one should have the opportunity to choose those two or three methods which cover those skills which are essential for the particular learner. In practical assignments, self-determined learners learner should have the chance to be trained and to practice precisely only those procedures they are interested in.

Another example that coursera is not interested in the self-determined learner is their change in the billing strategy: Coursera made a switch to a subscription model for the Data Science Specialization. Instead, to bill per course, you have now to pay a monthly fee, which is counteracting self-learning and autodidacticism. To economize and save money for the course, one has to study fast and without looking for additional material, exercises, or implementations for his/her use cases. Consequences are a rigid study behavior where students follow the course design uncritically and without generating and posing questions. From an educational point of view, such a course of studies contradicts the psychological research on learning thoroughly [@ambrose_how_2010; @brown_make_2014].

How to support the self-determined learner?

Consequently, I stopped my course attendance and started to look for tutorials, courses, and other learning material that are more to the point for my needs. I have to confess that I didn’t know how exciting but also how difficult this self-paced and self-determined learning approach was, respectively is. Between us educators, we are talking routinely about self-determined learning [@deci_handbook_2008; @deci_overview_2012-1; @deci_self-determination_2008-2], but often it is just a plea without detailed tips on how to achieve it.

When we are preparing material for learners, we are restricted by the learning time of our students (measured in credit points) and have therefore to select from the available resources. This strategy poses a problem for the self-determined learner as we as teacher decide which material is essential to learn and which not. Even if we are experienced teacher on the subject material in question and know therefore which skills are generally more in demand in real-life applications, we will always be wrong whenever some of our students have different needs or requirements.

For the design of online courses, there is an easy solution. Even if every curriculum has a reference point for the average amount of learning hours a student should need to cover a specific subject, there is no restriction on quantity for uploaded material. When we quantify the necessary learning hours of each different subject branch we are going to provide, then we can leave the choices to the students. The only requirement they would have to satisfy is a certain amount of learning hours. Students have to choose from a list of options in such a way that they fulfill this requirement by the curriculum.

To provide a course design where all students have to learn the same material generates an anachronistic situation: We are living in times of opulence where for almost all products, many variants for individual choices are available. But not so in the educational sector where with cohort learning still the industrial mass production model is prevalent. It is time to change this now — and with online courses, we have already the appropriate learning environment for individualization ready at hand.

Résumé

Online courses should not only address learner looking for (non-)formal education and validated certificates but should also support the self-determined learner with its particular learning goals. Yes, my recommended educational strategy in the case mentioned above is a laboriously and hard-working solution in two aspects:

You need to work out all different learning paths with their answers, examples, exercises, and tests.
You need to provide an introductory section with a basic explanation of each branch so that students can make an informed decision.

I agree that these two requirements for the self-determined learner are additional work. But for MOOCs with their vast numbers of learners from different backgrounds, an individualized educational design would be valuable and economically feasible. Another advantage of the suggested instructional method to teach all approaches for a specific subject (e.g., ‘How to get data into the R environment?’) is that every one of these different procedures to choose from can be explained in exhaustive detail. As the various branches of the learning material separated in different self-contained modules, it is easy to add new methods or correct out-dated ones. With the proper design approach, each of these modules is an autonomous learning object that can be used in other (learning) contexts as well.

References

Massive Open Online Course ↩︎

RStudio Snippets for Markdown

Wed, 12 Jun 2019 00:00:00 +0000

Tutorial for RStudio code snippets

Code snippets are text macros which you can insert into your programs. They are used for repetitive tasks and can boost your productivity tremendously. Here I am not going to explain how to use it. There are already several good tutorials around:

The official RStudio tutorial on code snippets
A Blog entry by Jake Daniels: How RStudio Snippets Improve My Productivity
A Blog entry by Jozef Hajnala: 4 ways to be more efficient using RStudio’s Code Snippets, with 11 ready to use examples
A Video by Sharon Machlis: R tip: Save time with RStudio code snippets (part of her video channel Do more with R)

Eight useful reminders

In this post, I want to summarize some important details, which I learned laboriously after time-consuming trials:

The Markdown section in the snippet editor works only with .Rmd and .Rmarkdown files. – So if you are going to write just text (e.g., a blog entry) and you want use RStudio snippets to insert shortcodes than you have to write into .Rmd / .Rmarkdown files.
The snippets in the R section of the snippet editor are for code chunks whereas the snippets in the Markdown section work only in markdown text of .Rmd / .Rmarkdown files.
To insert snippets in the Markdown part of .Rmd / .Rmarkdown files, you have to use SHIFT-TAB (and not just TAB as for R snippets in the code chunks).
To write your snippets, you have to use the TAB key for indenting. 4 Spaces do not work!
It is legitimate to write huge snippets for large code junks. For instance, you can write complex ggplot2 snippets as a kind of a template you are going to use frequently.
For large code chunks, it may be preferable to write the snippets with the RStudio code pane. You can load the snippet files under ~/.R/r.snippets respectively ~.R/markdown.snippets into RStudio.
You can share you snippets sending these files to your colleagues and collaborators.
Do not use names, which are difficult to type to call your snippets. For instance, to insert the markdown syntax for an image do not name it “![” but “ii” for example.

What is Open Science About?

Wed, 12 Jun 2019 00:00:00 +0000

The double meaning of ‘Open Science’

To understand the challenges Open Science poses to civil society, scientists and the public alike have to focus on the double meaning of this term. On the one hand, it refers to a movement to make scientific knowledge publicly accessible for everybody, and on the other hand, it strives for open procedures in the knowledge creation itself.

This second aspect (“to strive for open procedures in the knowledge creation”) is often not included in the definition of Open Science, or at least the significance of this goal is not appropriately valued. The following quotes illustrate our claim that the process facet of Open Science is frequently underdeveloped.

Open science is the movement to make scientific research (including publications, data, physical samples, and software) and its dissemination accessible to all levels of an inquiring society, amateur or professional. Wikipedia

Open data and content can be freely used, modified, and shared by anyone for any purpose. The Open Definition

The first example is from Wikipedia, the most popular website featuring not only free content but also supporting community building for Open Knowledge. The immediately following quote is the short version of the famous Open Definition by Open Knowledge Foundation, a global non-profit organization, dedicated to help civil society groups to access and use data to solve social problems.

Accountable for not mentioning of the process components of Open Science is not the compact form of these two definitions, as can be seen by a more exhaustive wording in the next citation. The following third example uses the term Open Knowledge instead of Open Science, but as explained further below we will not only show the equivalence of these two notions but argue that Open Knowledge is even the better concept.

‘Open knowledge’ is any content, information or data that people are free to use, re-use and redistribute — without any legal, technological or social restriction. We detail exactly what openness entails in the Open Knowledge Definition. The main principles are:

Free and open access to the material

Freedom to redistribute the material

Freedom to reuse the material

No restriction of the above based on who someone is (such as their job) or where they are (such as their country of residence) or their field of endeavour (including whether they are working on a commercial or non-commercial project)

Open knowledge is what open data becomes when it’s useful, usable and used - not just that some data is open and can be freely used, but that it is useful – accessible, understandable, meaningful, and able to help someone solve a real problem.

So open knowledge is empowering – it helps us effect change and improve the world.

Clearly, this an applaudable quote! It targets to foster social responsibility and civil society. But look at it in more detail: The focus is on ‘material’ (= product), there are no references to the generation processes of material. Taken verbatim it does not necessarily include Open Research Workflow, Open Methodology, Open (Peer) Review or Open Scholarship, to name just a few notions relevant in the making of knowledge. Knowledge does not sit around and wait to be picked up but emerges in a (controversial) construction process. (Latour, 2007)^[Latour, B. (2007). Reassembling the Social: An Introduction to Actor-Network-Theory (New Ed). Oxford University Press.]

Openness as conditia qua non for transparency

A better approach is, in our opinion, the definition by Foster, an EU funded project:

Open Science is the practice of science in such a way that others can collaborate and contribute, where research data, lab notes and other research processes are freely available, under terms that enable reuse, redistribution and reproduction of the research and its underlying data and methods. Foster

To embrace collaboration and contribution in the definition signifies that only access to the products is not enough and automatically includes the requirement that the whole research process has to be open to qualify as transparent.

What are the consequences of our insistence in transparent processes in all phases of the research endeavor? Transparency in its own right is crucial for reproducibility, a topic we will have to say a lot more later on. If one takes the double meaning of Open Science seriously, then it includes technology means (e.g., machine processing techniques) as well as behavior adaptation of institutions and scientists (e.g., social changes). If not only the product (the scientific findings) but also the process of knowledge production (the scientific workflow) has to take place without (social, technical, legal, etc.,) barriers, then research data and their interpretation must be transparent in every aspect. It covers the epistemological interest as the starting point, followed up by all kinds of manipulation and reprocessing until the findings get finally published.

Open Science versus Free or Libre Science

The openness in Open Science is not restricted to public access. Therefore, we cannot translate the term as ‘Public Science’ without losing some of its meaning. Maybe “open” is the wrong qualifier in the first place? Richard Stallman argues extensively that Open Source is not the same as Free Software (see for instance What is free software? or Why Open Source misses the point of Free Software). He explains that ‘free’ is not a matter of price like in “free beer” but of freedom like in “free speech.” Looking for a better word, he suggests to use Libre; a word borrowed from the French or Spanish language and where there is no confusion between ‘free’ and ‘freedom.’ Taken this line of reasoning into account: Perhaps we should also prefer to talk about “Free Science” or even better of “Libre Science” instead of “Open Science”?

Assuming a holistic perspective and taking all mentioned three elements together – access to the research products, access to the research process and the right (=freedom) to participate or collaborate – has significant consequences: It calls for a cultural transition with a modified research practice (Open Research Practice) and thus a new self-concept as a researcher (Open Scholarship). And last not least, we will not only focus on some technical improvements but also on the need to change the power relations in our society.

Open Science versus Open Knowledge

The English terminology of ‘Open Science’ has the disadvantage that with ‘Science’ is meant the natural sciences predominantly. And indeed, the Open Science movement is far more prevalent and entrenched in the STEM subjects (Science, Technology, Engineering and Mathematics than in HASS (Humanities, Arts and Social Sciences) disciplines. As a result of this observation, Michelle Sidler suggests:

The movement should thus consider changing its moniker to open knowledge in order to include academic disciplines that do not self-identify as science (2014, 77).

Although we doubt that the meager involvement of HASS in Open Science can be attributed solely to the chosen naming, Sidler’s suggestion is appealing for us. Even under the caveat that ‘Knowledge’ also has a double meaning in English: ‘Knowledge’ in the general understanding that some knowledge is already acquired and knowledge in the more philosophical sense of insight or discovery. In the first denotation, ‘Knowledge’ refers to the ownership of a product, in the second to the process of acquisition. However, from our perspective, this is just a desired ambiguity!

Open Science versus eScience

Another misunderstanding arises from the notions of ‘Cyberscience,’ ‘Science 2.0’ and ‘eScience.’

Although there are overlaps between Open Science and Science 2.0, we want to emphasize that these two terms are not congruent: Science 2.0 refers to collaborative processes using the so-called Web 2.0 and stresses, therefore, the elements of participation and sharing. It is legitimate to use the notion of ‘eScience’ when, for instance, researchers work together with appropriate cloud-based software on a scientific contribution. But if the resulting publication is not freely available, then we cannot speak of ‘Open Science.’

For Open Science, it is not decisive whether the research process takes place individually or collectively, but whether all activities are transparent and freely available and changeable. Yes, web-based technologies facilitate free availability, access, and transparency, but we should not confuse the possibilities and properties of tools with their final product. (See similar reasoning in Bartling and Friesike 2014, 8^[Bartling, S., & Friesike, S. (2014). Opening science: The evolving guide on how the internet is changing research, collaboration and scholarly publishing. Springer.]; Leibniz Research Alliance Science 2.0 2016^[Leibniz Research Alliance Science 2.0. (2016, May 27). Aim and scope | Leibniz Research Alliance Science 2.0. Retrieved October 8, 2018, from Leibniz-Forschungsverbund Science 2.0 website: http://www.leibniz-science20.de/ueber-uns/aufgaben-und-ziele/]).

Wrapping up

We have discussed several elements and approaches to defining Open Science. It turns out that this notion suffers from some inherent and implicit shortcoming. It would be better to use the term ‘Libre Knowledge’ to avoid possible misunderstandings. But as the concept of ‘Libre Knowledge’ is not widespread and in common use, for the sake of simplicity we will apply ‘Open Knowledge’ or even ‘Open Science,’ but in the broader meaning as we have outlined above.

Grammarly: A Case Study

Sun, 09 Jun 2019 00:00:00 +0000

In another post, I reported on my general experiences with Grammarly. I commented on different use cases, mainly with text pieces from my blog writing. By contrast, this post here focuses on a case study with a literary text piece from a novel.

I have taken the first 5,000 characters of a still unpublished German novel from a friend of mine. For an informed assessment, I am not just going to explain the different steps but will also provide the resulting text documents as attachments.

The different phases of my workflow

I ran the text passage through Google Translate and revised the text with Grammarly in two steps.

Figure 1: Screenshot of Grammarly alerts after Google Translate

Step 1: This is the German original text passage.

Step 2: The text after Google Translate (1 minute). In several passages, it is not understandable, and it feels awkward and clumsy.

Step 3: Working through the text with Grammarly uncritically, e.g., without reading and thinking, only following Grammarly’s alerts (2 minutes). Grammarly comes up with 19 alerts, almost half of them about spelling (5) and punctuation (4). Grammarly does not detect the incomprehensible sentences or clauses.

Step 4: Working through the text with Grammarly thoroughly, e.g., finding (with dict.cc) better suited vocabulary and changing sentence structures where appropriate to get a more correct and lively English (75 minutes).

I believe the final text is much better; even it is still imperfect and flawed. Please keep in mind that I am not an English native speaker.

The essential conclusion I want to draw after this experiment: The resulting text passage is in a quality I would have never achieved without the three tools I mentioned: Google Translate, dict.cc, and Grammarly].

Figure 2: Screenshot of Grammarly plans and pricing

Comments desired!

I would love to get comments from you:

If you are a German native speaker: What do you think about the quality of the result concerning the time needed?
If you are an English native speaker: Would you confirm that the Google Translator text is clumsy and in some passage not understandable? And first and foremost: Is the quality of the final version not all that bad and tolerable?
If you are neither a German nor English native speaker: What do you think about the final result respective to your English language skills? Would you think that Grammarly could improve your English?
And to all of you: Did the final text passage entice you to read more of the book? Should my friend invest 140 bucks to buy the premium version of Grammarly for one year and spend her leisure time to publish her novel?¹

Just a rough calculation: For 5,000 characters I needed all in all about 80 minutes. My friend’s English is better so that she would be possibly faster (and better) in the translation. But let us continue the calculation with 80 minutes. The novel has 450,000 characters = 90 x 5,000 character, i.e., 90 x 80 minutes = 8,100 minutes or 135 hours or 27 working days á 5 hours. ↩︎

Grammarly improves your writing

Fri, 07 Jun 2019 00:00:00 +0000

My story with Grammarly

This article reports on my experiences with the premium version of Grammarly, an AI-powered web service for grammar and spell checking. As this is going to be a very positive review, I feel the obligation to publicly disclose that I am not in any way affiliated with Grammarly Inc., the enterprise behind this product.

It was almost exactly one year ago in June 2018 when I learned the first time about Grammarly. I cannot remember anymore, why I stumbled over Grammarly. At first, I used it only with the free plan experimentally for some of my English writing on the web. I have to confess that I was very skeptical about the practical usefulness of this kind of app.

But after some tests, I was surprised about the quality of the suggestions. Grammarly was quite helpful as it gave me not only corrections for words spelled wrongly but also tips and ideas for a better style in English writing. At the same time, Grammarly told me not only how many improvements it has already suggested to me for a specific text passage, but also how many more suggestions would be in the premium version. Clearly enough, this was a sales gimmick, but a well-done one! I became curious about these other recommendations and signed finally up in August 2018 for the Premium version. At that time, it was still US$ 62,98 for a year, but in the meantime, the price has more than doubled (US$ 139,95) for the yearly subscription. I didn’t regret it!

Use cases illustrated by screenshots

Besides spell and grammar, Grammarly also helped me with style issues and even to find the right tone to address a particular audience. As a non-native speaker, it is difficult for me to judge any English improvements. But from reading much English literature, I’ve got the feeling that my English writing has improved tremendously.

I do not intend to write a tutorial for Grammarly, but I will give you some examples of use cases, illustrated by screenshots.

Use case 1: An already published article

I took a lengthy article (about 2.500 words), which I had published two years ago. I copied the markdown formatted text directly into the web interface of Grammarly. This article has a somewhat philosophical flavor so that technical concepts, which Grammarly has not in its dictionary, didn’t play a role. I had the text already cleaned up form spelling errors using RStudio’s spell checker. Nevertheless, Grammarly came up with more than 125 suggestions for improvements. I didn’t think that my English is so bad. How embarrassing!

When I inspected the text in detail, it turned out, that 19 suggested “improvements” weren’t useful, as they addressed quotes from English books by famous people like Michael Polanyi or Ludwig Wittgenstein. Another four recommendations related to markdown formatted text, where corrections would have resulted in syntax errors. But still there remained about 100 helpful tips! Statistically, Grammarly advised me about possible improvements every 25 words. I was shocked, and even a little depressed!

Grammarly suggests and explains a possible improvement

You see on the right top of the screenshot the number of alerts or warnings (= 120). Grammarly suggests finding a more common synonym for “emphatically” and explains with examples why it is maybe not the best choice. As all recommendations didn’t quite fit for my writing intention, I decided to delete this “overly complex” word without a substitution.

Use case 2: German text translated via Google Translate

I transcribed a German interview with Michael Rundel and ran the text through Google Translate. I believe, Google Translate has already overcome its start-up difficulties. Nowadays, I think, its translation is pretty good. But the writing must still be reviewed by a human.

As you can see in the image below, Grammarly recommended after an unpolished Google Translation from German to English a slightly better rate of improvements per word: 1892 words (bottom-left) with 104 tips (top-right) which accounts for one suggestion every 18 words.

Grammarly alerts in a machine-translated text

If you subtract the high number of 37 tips for punctuation issues (third line in the alerts list) – which partly are markdown formatting requirements and partly are differences between American and English punctuation rules – then the ratio improves to 1 tip for every 28 words. Admittedly this calculation is done under the caveat that after Google Translate, one has more sentences to restructure or to reformulate completely. But you can use Grammarly to implement these changes fairly quickly.

This calculation startles. Instead of struggling to formulate an English text from scratch and then to polish it with Grammarly, it appears to be more efficient – at least for me – to use another workflow: Writing my thoughts in a German paper -> than to translate it with Google Translate -> and finally using Grammarly to polish it.

Write the text in German

Translate it with Google Translate

Polish it with Grammarly

Publish it with Bookdown /Blogdown

But this reasoning is not entirely correct: The first version of the English text could already be done with Grammarly and would save one complete phase of the mentioned workflows. At least, I believe, this is valid for technical writing, where writing in English is not so complicated grammatically and therefore faster. But for publications in the Humanities, the workflow with Google Translate would be perhaps the better choice.

Use case 3: Writing English directly with Grammarly

To write text “live” in Grammarly has several advantages:

Grammarly gives you immediately a short explanation for every alert so that you can reformulate or restructure your document instantly.

Grammarly explains every current alert briefly and succinctly

Grammarly offers on-demand also longer clarifications and examples. So you can learn better writing during writing.

If required you can get from Grammarly a more profound explication with examples

Grammarly tutors you about the English language. This educational approach comes in handy when you notice that Grammarly often complains in similar situations. For instance, I didn’t know that there are different punctuation rules in American and British English. As I had written the lengthy text outside of Grammarly, I had to go through the whole article and change all occurrences of wrong punctuations one by one. If I had known these different rules, I would have to use the correct rules already in writing the draft version.

Grammarly offers on some subjects short tutorials

Grammarly supports different kinds of workflows: In the default configuration, Grammarly jumps automatically from one suggestion to the next. It is a good strategy for papers already written in English. But if you are writing directly within Grammarly, it is better to turn off this feature as it destroys the writing flow. Then you can select the recommendation you want to look at individually, and Grammarly jumps to the related text passage. A third option is to work on all tips concerning a specific category (e.g., spelling, grammar, punctuation, clarity, etc.) As you don’t have to jump from one type of problem to another one, the focus of one category of alerts speeds up the correction.

Working with Grammarly on alerts type ‘Punctuation’

Grammarly opens a window with synonyms when you double click on a word in your text.

Grammarly window to choose from a suggested synonym

Grammarly also offers recommendations to improve your writing style. The following screenshot presents my first try to begin this article. Grammarly analyzed it as a tedious text passage. This example demonstrates that Grammarly is far more than just a garden-variety of a spell or grammatic checker.

Screenshot of a Grammarly window with alert about a boring text passage

Grammarly varies its suggestions according to your writing goals. You can choose your intention, subject domain, target audience, your writing intention, your style (formal/informal), and even about the emotional flavor of your writing. Some of these features are still experimental, and I have to confess that I did not know when to choose what. I assume that my English is not good enough to use these advanced configurations.

Grammarly varies its alerts according to your chosen writing style

Grammarly can seamlessly be integrated via a browser plugin into types of web content (e.g., fora, web editing). But it seems to me that these possibilities are sparse. Yes, there is an add-in for Microsoft Word, but only for Windows. In WordPress, Grammarly does not work with the visual editor, you have instead to use the code editor. Likewise, you can’t use Grammarly with markdown files in RStudio or other text editors. But these shortages are not limitations as you can always write your book or article in the Grammarly cloud and then copy the text into your editor of choice. Grammarly stores your document in short time intervals, so you don’t have to worry to loose parts of your work. You can go back to unfinished articles at a later time and finish it for publication.

My personal desktop in Grammarly

I prefer to work with the browser extension, as it always presents the same user interface. Besides, it unleashes the full power of Grammarly. All screenshots in this post show the Grammarly editor in the browser, Google Chrome, in my case. But there are also plugins for Firefox, Safari, and Edge. All these browser extensions are free, and you can use Grammarly with limited functionality at no charge as well.

Other similar products

Finally, I should add that during the research of this article, I noticed that there are a couple of web services with related functionality available. The high number of recent articles about grammar apps insinuates a big market for this kind of service:

Wrapping up

All in all, I’m delighted with Grammarly. In my opinion, it is a beneficial tool for the non-native writer. It improves your writing for the web (e.g., posting in blogs or commenting in fora) but also for printed papers and books. It has many cute and even some outstanding features. Even with Grammarly, I am still not wholly comfortable to write in English, but at least I am stressed about it anymore.

I do not claim that Grammarly is the best product, as I do not know the other apps and have therefore no comparison. But in most of the reviews mentioned above, Grammarly is top ranked. It is undoubtedly the app with the biggest user base (over 10 Mio.).

Last, but not least, I would like to add that Grammarly is not idiot-proof. You have to make your choices, looking up other dictionaries to find better wording. (I am happy using all the time dict.cc). You have to be aware that even if Grammarly is AI driven, it does not understand your writing in the human sense. If you are writing a dumb and meaningless article, Grammarly cannot improve your reasoning and arguments. I noticed, for instance, that Grammarly sometimes had overlooked a missing word, an error a human would immediately see. And sometimes Grammarly is wrong with its suggestions as the last screenshot demonstrates. But these rare cases are easy to judge and correct.

Grammarly is not idiot-proof. Your own intelligent judgement is still required.

Physik Libre: A bookdown project

Wed, 05 Jun 2019 00:00:00 +0000

The article is an interview with Michael Rundel. Michael is a teacher at a grammar school in Vienna/Austria and teaches physics, computer science, and media design. He talks about his experiences with bookdown in writing Physik Libre ( https://physikbuch.schule/), a new textbook on physics. His project is a practical implementation of cross-media publishing with bookdown, which I had described in my 7-part German tutorial on my other blog Gedankensplitter.

Status of the following interview

This article is an interview with Michael Rundel. Michael is a teacher at a grammar school in Vienna/Austria and teaches physics, computer science, and media design. He talks about his experiences with blogdown in writing Physik Libre ( https://physikbuch.schule/), a new textbook on physics. His project is a practical implementation of cross-media publishing with bookdown, which I had described by in my 7-part German tutorial on my other blog Gedankensplitter.

Michael Rundel (CC-BY Michael Rundel)

The interview took place two years ago, but I didn’t find until now the time to translate it into English and to publish the article in this weblog. I asked the questions in German via email and translated the text with the help of Google Translate, dict.cc, and thesarus.com. Where it was useful, I have links provided the first time the concept or software was mentioned. Subsequently, I have these same notions marked as red text. I edited the article with the premium version of Grammarly. If you are a native English speaker, I would be happy to know about the mistakes in grammar or other language issues. Please use the Edit Page button on the top right corner to provide improvement via a pull request to my GitHub repository.

I’ve gotten to know Michael during one of my talks about the free software environment for statistical computing and graphics R at Danube University Krems. In this talk titled “The many faces of R,” I demonstrated to (mostly computer science) teachers the many different use cases of R, driven by special R packages like bookdown. When people wanted to see a practical demonstration for school purposes, Michael came to the front and presented Physik Libre. I was delighted not only to have a school-based example but also to find a colleague near my hometown who is also working with bookdown.

Why another physics book?

Question: There are so many physics books why write one more?

There is already a very long tradition of Open Education Resources (OER) in the US. A couple of years ago I found English free physics books for study and teaching (for example Physics for K-12 (Sunil Kumar Singh), People’s Physics Book, CK-12 Physics - Intermediate, …). In German-speaking countries, there was something like that not yet. In the meantime, I have found with Physikunterricht Online an exciting project in the German-speaking area. But in my opinion it does not go far enough: There are more possibilities for interactive usage nowadays. And I want to build the most state-of-the-art physics book available for students.

We can see many extremely positive developments in recent years: web technologies have matured, projects like Gitbook allow you to create documents with minimal effort, we now have a variety of free program tools and Creative Commons licensed more than 1.1 billion media available on different websites and repositories for educational purposes available , and collaborative platforms like GitHub, GitLab or Bitbucket are no longer used exclusively by programmers.

Physik Libre as “Proof of Concept”

With Physik Libre I want to show what is possible today with the latest technology using the example of a physics book for the secondary school. It is a proof of concept – if you will. The last few years of Linux have shaped me, I’m afraid. The following points are important to me personally for the implementation of the project:

Page of ‘Physik Libre’ on a smartphone

Device-independent (Smart TV, PC, notebook, tablet, smartphone, e-book reader, print)
Operating system independent (web browser, ePub reader, PDF reader)
Free Standards (HTML, JS, CSS, ePub, PDF)
Free Licenses (CC-BY-SA 4.0, GPL 2)
Free Software supporting online publications like bookdown, GitBook, R, pandoc, LaTeX, LyX, Inkscape, etc., sharing all their material on free accessible repositories (GitHub / Gitlab / Bitbucket)
Free access and respect for privacy (no accounts, no registration, no tracking)
Contemporary (full-text search, sharing function, etc.) multilingual approach.

Question: What was the origin of your idea to write an online book?

Since my students days in the 1990ties, I have tinkered with web technology. Although I was always impressed what is achievable by combining HTML+JavaScript I never dreamed of the possibility of building full applications like Google maps or tinkercad by using pure web technologies.

In Austria, the current so-called digital schoolbook of today is a PDF document put on the internet. There is much more possible with today’s web technology. Much more!

Question: Could you give me a short overview of the historical development and the different phases of your project?

The idea was roughly born three years ago when I was experimenting with the canvas element, which I was using in my physics applets. This technology helped me to abandon flash applets. Don’t get me wrong! – Flash was a neat tool in its time, and I loved working with it. But it always felt like a workaround, and furthermore, it was proprietary software. At that time (2016), I thought there are all building blocks available to realize such a project with pure web technology.

Cover of the online physics book ‘Physik Libre’

The following months I learned [SVG] (https://en.wikipedia.org/wiki/Scalable_Vector_Graphics) and Inkscape, discovered Markdown and the document converter pandoc, the version control system git and the GitBook project. Summer 2017, I learned about the R package bookdown. The following school year, I have taught a class of 5th grades of Austrian secondary school (second year of the upper school) in physics. So I decided to start the work on the project.

Question: What were your most significant hurdles to overcome in finishing the Physik Libre project successfully??

The biggest challenge for me was time constraints. I managed to finish the project with sleeping sparsely.

Question: Were there some favorable circumstances for the project?

First of all: I would have never had a chance to do such a project if there were not such excellent tools like the Linux software and the combined efforts of the Wikimedia community.
On the technical side: I studied Mathematics and Physics, but I also have acquired some essential programming skills. This knowledge helped me to glue together some parts in the workflow of the book.
And finally on the personal side: First there is the love of my wife and my child. Also, my school principal is so generous as to give me a share of our schools web space for my project.

Question: I noticed that you deliberately do not call your book a textbook. What’s the difference between your physics book and a physics textbook?

A certified textbook always has to be based on the current syllabus and exam regulations. It contains competency-related tasks and guides for teachers. Most of the books websites – at least in the German-speaking countries – are not yet approved by the Ministries of Education. But they play an essential role as a “second opinion.”

In this sense, I understand my physics book project as a “second opinion” for teaching physics to high school students. It tries to explain connections as well as possible – and therefore also with all technically feasible means – for the mathematical level of a high school student.

Question: You have a full teaching commitment and family. How did you come up with the idea to approach such a mammoth project as an individual?

At the beginning of the project, two considerations were crucial.

First, everything - except for the actual writing of the text and the occasional drawing of graphics (where I could not find suitable content on Wikimedia Commons) - should be automated via programs.
The second point is not to view the book as my proprietary, but to view it as a collaborative project. All source files of the project should be hosted on a platform like GitHub or GitLab, thus minimizing the hurdle for other people’s participation in the book and encouraging them to participate –even if they find and correct just a spelling mistake.

But real life is quite different!

Page of ‘Physik Libre’ on a desktop computer

Concerning the first point: Although there are already many excellent free tools such as pandoc and bookdown for the generation of documents and final formats, the devil is in the details. For example, many images from Wikimedia Commons, have wrong dots per inch (DPI entries in the metadata. But this piece of information is essential for printing. If I use, for example, an animated gif picture in the web version, a particular frame must be captured and integrated for the print version.

For all such cases, I had to write programs that automate these tasks.

Reflecting on the second point, I mentioned I have to confess there is still a kind of Wild-West mood in the project: Until now I experiment a lot what is feasible. However, this also means that this additional functionality will change the program. Therefore it is challenging to find collaborators when even I do not know what the final features will be. For fruitful cooperation, one needs to establish a common goal with a framework that is the most salient aspects already consolidated.

I taught the fifth grade in physics last year and tried to write the content in the book in parallel with my lesson preparation so that my current students can benefit from my progress as well. So far, I have mostly succeeded, but it is very time-consuming, especially since I am – unfortunately – not a natural-born author.

Plans with Physik Libre

Question: It sounds that your book is still a work in progress. What are your plans for further developments?

I have already implemented solutions for most of the technical issues I have encountered. But these program snippets are not very elegant, and there may be a more straightforward answer - after all, I am not a professional programmer.

One problem I am still struggling with is the same-origin policy concept, which limits the embedding of content from other web sites. So far, I have not found a satisfactory solution to this problem.

Besides, much work needs to be done on the documentation before I want to release the project on Github or GitLab.

Thank you, Michael, for this interview!

Comments in Markdown

Sun, 02 Jun 2019 00:00:00 +0000

In reworking this docdock-theme, I wanted to document the reasons for my changes and how I have done it. But I didn’t know how to write comments in markdown.

Yes, one could use a normal HTML-comment, but you still can see the comment in the source code.

<!-- 
This is a comment inside a code chunk, so that you can see it on the page.

Multiline comments are allowed as well 
-->

Look at the source code, and you will see the text example below this line.

Searching in Stack Overflow I came finally up with the following solution:

[comment]: # (This text is a comment! But written in a code junk, so that you can see how it is done.)

[comment]: # (This text is a comment! Multiline comments are allowed as far as long there is no line break. This text is a comment! Multiline comments are allowed as long as there is no line break. This text is a comment!  Multiline comments are allowed as long as there is no line break. )

You can’t see the result of this last code example not even in the source code. The comment is hidden by definition :wink: But you can inspect the original text file on my GitHub repository. Just click on the Edit page link in the top right corner of this page.

There are also other possibilities, but the above solution with comment: written in square brackets and the # - sign followed by the comment written inside round brackets is the most portable version.

Blog writing barriers to overcome

Fri, 31 May 2019 00:00:00 +0000

Another year has elapsed since I wrote my last blog entry. At first, I thought the problem is only my English. But in the meanwhile, I see several other reasons why I do not blog for the new static website(s).

I will elaborate later on the following thoughts. At the moment, I want to list the possible reasons for my blogging abstinence:

I am still not comfortable with the new community I want to address with these blog entries.
The topics I want to cover are still relatively new for me so that I often believe that my reflections and discussions of these subjects are trivial and not worth reporting.
I am doubtful which of my different upcoming ideas belongs to my German weblog “Gedankensplitter” and which is better suited for the English community.
I am still struggling to get the focus right for my English blogging. In the last few months, I get a better feeling on which two topics I would like to concentrate: Open Science and Data Science. (More on these two topics and their relationship will follow in another blog entry.)
From my professional point of view, I still generate most of the experiences and material in German such Interviews, Reviews, Talks, Lectures, etc..
It is much more exciting for me to learn new things in R or about strategies for Open Science than to continue to blog frequently for my German weblog.
There is not enough time to write for both weblogs. I wonder how other people like Andrew Gelman write at least one blog entry every day, sometimes even 2 or three.
Coming closer to my retirement the necessity to foster my professional online presence is not so vital anymore.
The English language is also a barrier. I am not so comfortable and skilled to write in a witty and informal style as I am used to it in German.
And finally there are these enormous complexities with blogdown and Hugo. There are so many software layers to watch out with all their difficulties arising for updates, incompatibilities, and bugs. As I have not used these technological infrastructures on a daily bases, it happened that I either forgotten many details. Instead of writing down my thoughts, I had to waste my time to re-learn the software tools.

As you can see: A considerable bulk of difficulties and barriers to overcome! Some of the points of this list, I will reflect more in detail at a later time.

ORCID - Researchers uniquely identified and connected

Sun, 26 Aug 2018 00:00:00 +0000

Do you know ORCID? I applied for my ORCID-ID already several years ago, but only yesterday I understood the advantage of yet another web presence: ORCID is a non-profit organization and aims to identify researchers uniquely and to connect their contributions and affiliations across disciplines, borders, and time. With this post, I tell you some advantages of ORCID to convince you to join.

What is an ORCID ID?

The last two week I worked hard to finish the ‘official’ nitty-gritty this web presence: CV, downloads for publications, disclaimer and privacy policy. By this occasion, I cleaned up my .bib files and subsequently my Zotero and ORCID presence.

The goal of ORCID is to provide an identifying infrastructure for individual research activities. The main idea is to connect research, funding, and publishing institutions with the individual researcher via a unique ID. It is similar to an electronic portfolio where you can establish so-called ‘trusted’ connections, e.g., where institutions are allowed to look into your scientific record and to add information to it. These institutional connections are — by the way — also the funding structure of ORCID: Membership organizations have to pay a pretty high subscription fee (starting with 5.150 US $) to connect and use their infrastructure with ORCID.

However, this is not very important to me. At the time of this writing (2018-08-26) there are solely 910 member organization, only four of them from Austria:

The Austrian Science Fund (FWF)
Technical University Graz (TU Graz)
University of Natural Resources and Applied Life Sciences Vienna (BOKU) and
University of Vienna

Only TU Graz has already established integration with ORCID. The other three universities are still in the process of planning the technical infrastructure for this bonding.

A personal, digital, international and persistent name identifier

However, ORCID is already very advantageous for the individual researcher: The ORCID ID gives you a personal, digital, international, and persistent name identifier. This ID outlives the change of working positions with their associated web presence and email address. More important: The ID identifies you unmistakable, e.g., independently name changes with marriage or a pool of people with precisely the same name.

QR Code for my ORCID ID

Especially the last case is crucial for me: I know from a funny event (1st International Meeting of people with the family name of ‘Baumgartner’) that there are about 4.700 (!) Baumgartner’s in Austria alone and perhaps several thousand more in the world. As my given name ‘Peter’ is also trendy, there are always a bunch of Peter Baumgartner’s in every database. Even when I want to redeem a voucher in a small shop, I have to provide my address as there are two or three names with “Peter Baumgartner” in their database.

This coincidence of names is very cumbersome when somebody tries to find my publications: There are way too many publications under my name! And not in all cases, you can decide and separate them with adding the research field.

There is especially another Peter Baumgartner in Computer Science situated for many years in Koblenz (Germany). As I am working on E-Learning issues, you can imagine that there arose many times confusion: Sometimes we got even E-Mails for the other person. This Peter Baumgartner has now changed to Australia, but this has not helped much to clear up the situation as the confusion between Australia and Austria is legendary.

If you Google my name, you will find several others Peter Baumgartner, especially a top-rated speaker and coach: Peter Baumgartner, der Mutmacher (Peter Baumgartner ‘the bolsterer’ from ‘to bolster somebody up’). ORCID helps to prevent precisely this mistaking identity!

Other similar initiatives

There are other similar initiatives like Scopus Author ID and ResearcherID.

ResearcherID (developed by Thomas Reuters and used in Web of Science) and Scopus Author ID (generated by Elsevier and utilized in Scopus) are two examples of these efforts. Whereas ORCID is “a platform-agnostic identifier,” ResearcherID and Scopus Author ID connect to proprietary, subscription-based systems. (Quoted from the Library of University of Chicago)

Remains to mention: ORCID is free, platform independent and Open Source under the MIT License.

Screenshot of my the upper part of my ORCID page

Conclusion

ORCID aims to be a part of a broader social and technical infrastructure for uniquely identifying researchers with their work, affiliations, funding, and publishing institutions. Almost 5.2 Million researchers have already applied for their free ORCID ID. In the long run, many institutions engaged with research (universities, funding agencies, publishers) should be connected and form part of this global infrastructure. Even if this is not the case at the moment, applying has already some advantages: It gives you another place to present your work, e.g., a free web presence and provides you furthermore with a personal, digital, international and persistent name identifier.

Xaringan presentations with blogdown

Sat, 21 Jul 2018 00:00:00 +0000

How to integrate Xaringan in blogdown?

I am following in this short post the detailed description by Tim Mastny in his blog entry Embed Slides in Your Blog. Please read his article — especially the second half of it, where he is explaining the installation procedure.

Refer to an external slideshow

knitr::include_url('https://timmastny.rbind.io/slides/first_presentation')

Refer to an internal slideshow

knitr::include_url('/slide/xaringan-demo.html', height = "505px")

Summary

Both examples above are the same as in Tim’s article.

Two observations:

There is no need to load the Xaringan package with library(xaringan). At the moment, I do not understand why this is not necessary.
In the second example, the gray left and right margin have disappeared. The parameter height = "505px" does this. To tweak this, I had to experiment with the optimal pixel value. (Maybe there is a better way to tweak this parameter with the valid value, e.g., taking the value from the CSS.)

The height parameter solves one of the problems mentioned by Tim:

I’m also investigating to see if I can trim the iframe, so I only see the slides and not the gray bars on the side.

How to present slides in full-screen view?

Here I wanted to address his second issue:

I’d like to be able to use a relative reference to the slideshow’s HTML file in the static/ directory, but I think knitr::include_url() requires a live URL. I’ll have to do more research.

My first try: Using Markdown image links

My first try was: ![](/slide/xaringan-demo.html/#5)

It generates a kind of thumbnail. Maybe this is theme dependent. I once tried it out in another theme and got other sizes, e.g., over the whole width of the text column. Presumably, this has to be done in one of the CSS files of the corresponding template.

A kind of a thumbnail was not the result I was looking for, but it could become handy anyway. For instance, as a reference in an article. With the slide number behind the URL, one can reference easy every slide frame directly.

My second try: Using Markdown with absolute links

The second try is a link in standard Markdown to the file. It opens up the full-screen presentation. The code:

The second try is a [link in standard markdown](https://portfolio.peter-baumgartner.net/slide/xaringan-demo#1) to the file.

This is not ideal one always needs an internet connection for the presentation.

My third try: Using Markdown with relative links

The third try is a relative link in Markdown to the file. It opens up the full-screen presentation. The code:

The third try is a [relative link in Markdown](/slide/xaringan-demo.html) to the file.

Summary

This is the solution Tim was looking for. I tried several approaches in vain. Then I remembered that there is a demo blogdown-static website by Yihui Xie. There I found the solution!

Summary and conclusion

This third approach solves the second question mentioned by Tim. And this is an excellent solution: With a simple relative link in Markdown can everyone present (Xaringan) slideshows without an internet connection in full-screen view.

Sure there would have always been the possibility to go via the operating system to the file on the hard disk directly. But this is not very elegant and would complicate to present an integrated presentation via browser with other (multimedia) elements. One reason out of 11 why Xihui Yui hotly favors remark.js is the feature that you can press the back button in the browser, and you will go back to the page from which you visited the slides, instead of reading the slides in the reverse order again.

The next thing on my agenda is to learn how to apply Xaringan for everyday use. I have to read and try out:

the example slides (Xaringan and remark),
the readme file at the remark GitHub site and
the documentation in the remark wiki.

Then I will transfer — as a first real live application — my slideshows written with reveal.js stored at my abandoned site at https://notes.peter-baumgartner.net/slide/ to this new site here.

To inspect the full code of this post follow this link to my GitHub repo.

Using themes with blogdown: Lesson learned

Thu, 19 Jul 2018 00:00:00 +0000

Report from an Odyssean Journey

After a long interruption …

After almost one year of interruption, I started re-using blogdown again. But instead of writing new content, I had to struggle once again using the sophisticated machinery of Hugo and my academic-theme. With painful experiences, I learned that one has to be cautious with updates to prevent breaking changes. In this post, I will report on my Odyssean experience and — more important — I will suggest guidelines on how to start, explore, and use themes in blogdown.

Finally — after almost one year of interruption — I could reorganize my working responsibilities to spend more time working with blogdown and Hugo.

But (re-)starting blogging using these tools was not easy after such a long period. I did not only forget many procedures but also ran into compatibility problems with the necessary updates. It turned out for me that the R Markdown ecosphere produces not only valuable and powerful tools but is a continuously changing complex still stricken with some fragilities.

After reporting on these difficulties, I will summarize the lessons I learned.

I had problems with my old theme

Last year I was intrigued by the potent docdock theme. It has many dynamic features (aka shortcodes) and is uniquely good suited for complex structured information. I have learned in my posting career with WordPress on Gedankensplitter that over time, new subjects emerge and require a change of the content structure. I was especially delighted that docdock could produce dynamic HTML slides with the reveal.js framework, which I planned to use for tutorials and teaching.

After updating my software machinery (R, R packages, Hugo), I also installed a new release of the docdock theme. And here the odyssey started! I got many error messages which I could not solve. One problem was that I tweaked the theme (sacrificing several days of my summer holiday last year) without the necessary profound knowledge of the way Hugo works. Another problem was that after not using blogdown such a long time, I had forgotten many procedures of the interplay between blogdown and Hugo.

After several days of work, I did not manage to get a functional website. Worse! Meanwhile, I had destroyed my old (functioning) website as well. 😰 This disaster was for me an involuntary learning occasion to dive into Git and GitHub to save my work and restore my repo. I had not worked on a sub-branch and had carelessly committed changes that destroyed my web presence.

DocDock theme still maintained?

To avoid similar issues, I started a clean installation of blogdown with the docdock theme. This time I did not get error messages, but experienced another surprise! The theme style I preferred had other (new) problems: The display of code snippets was not correct anymore and destroyed the page layout. I reported this issue to the theme author. Only now I noticed that the author does not respond to questions. On the issue page of the docdock repo, I found the following message from another user:

Is this theme still maintained? Issues and PR’s seem to indicate not. I’m looking to add a documentation theme, but I’m not interested in adding one that’s been abandoned. (Issue 151)

This question is now unanswered for more than 20 days.😡 I do not blame the theme author. People have to work for a living; they donate their work and experiences to us whenever it is possible. But how could I be so naive only to judge my theme selection by the functionality of the theme? Not taking into account

the sustainability of the theme, e.g., its complexity related with its life cycle (updates, bugs),
the responsiveness of the author,
the community of people using this theme and helping each other!

At that moment, I abandoned the docdock theme heavy-heatedly. I had put so much work into its adaption! And I like its functionality and design! Yes, I could still use my working version, but then I will lose future Hugo features. I didn’t trust this theme anymore.

Problems finding another suitable theme

From docdock to ghostwriter

Now I decided to choose a more simple theme and started with the recommended theme list in the blogdown book. The only theme I liked from its design was ghostwriter. I installed it without problems and began using it. My first project was to learn the Xaringan package in combination with blogdown. I stumbled about it reading the new R Markdown book. It could be a way to replace my beloved html5 slide functionality from the docdock theme. Instead of producing slides using reveal.js, I had now to learn with remark.js another tool. But Yihui Xie — the author behind all this fantastic stuff in the R Markdown ecosphere like knitr, blogdown, bookdown, xaringan, etc. — explained convincingly why remark is preferable to other slide frameworks (e.g., slidy.

To shorten a long story: After several hours trying to bring Xaringan to work — I followed the simple synopsis by Tim Mastny in his blog entry Embed Slides in Your Blog — it turned out that in the ghostwriter theme files in the static folder are not rendered and uploaded. This insight came as a big surprise for me! I thought that it is a standard procedure that all files under the static/ directory are copied to public/ when Hugo renders a website. Only when I found an entry by Amber Thomas on Stack Overflow — one of the co-authors of the blogdown book — I learned that this could be possible.

The problem was — at least for me — a tricky one. The website worked locally but not when I deployed it with Netlify. Even when I noticed that there were no static files uploaded in my repository, I still thought it was me, who did something wrong. As a novice user, I always believe that the problem is on my side. But after many fruitless trials, I came up with the idea to try it out with another theme. So I installed the Xmin theme and learned that everything I did was done correctly.

From ghostwriter to Xmin and Academic

I abandoned the ghostwriter theme much more relaxed than before the docdock theme. I was not yet emotionally linked to the theme and had (thankfully) not started with adaption work. I learned from this experience that it is an excellent strategy to test things out with the two minimal themes Xmin or Lithium, which provides a stable reference point.

But how to proceed now? Should I stick with Xmin to be on the secure side? But I do not like this theme from its appearance. Also, my CSS knowledge is feeble, and I do not want to spend much learning and working time to adapt its design.

So my next choice was the academic theme. Last year I had already experimented with it, and it is one of the (more complex) recommendations in the blogdown book. It is a very elaborated theme with fantastic features designed for academic usage. But precisely or targeting the academic community, was the reason why I decided last year against this theme.

I am looking forward to my retirement, and I am fed up from all the administrative university business: The thought of (re)presenting all details of my scientific work and especially to transfer more than 120 publications from my WordPress blog to the blogdown academic framework made me furious. All I was looking for was a decent blog framework open for all the helpful tools from the R Markdown ecosphere.

But incidentally, just at the same time, I noticed problems with some of the WordPress plugins dynamically serving my list of publications. Besides, I understood that I am not forced to use all sections of the academic framework: I could focus on the posts section — and maybe add later on some other parts.

So I returned to my abandoned experiments with the academic-theme as I already did some configuration and had even written some content. First of all, I updated it to the new version and … — Yes, again I ran into troubles! After some trials, I noticed a wired problem: It took me two days to narrow down the error. It turned out that with one of the last update, the academic theme was not compatible with blogdown anymore! I reported this issue to the RStudio community, to George Cushen and Yihui Xie.

Can you imagine my emotional state? I changed between desperation and depression. All in all, it took me already two weeks with the result that I had accomplished nothing!

Crazy Hair GIF from Crazyhair GIFs

Figure 1: Crazy Hair

After taking a short nap to calm down, I reconsidered my situation: Yes, it took me much time, but at the same time, I learned a lot. I am not referring primarily to my speed generating a new blogdown website from scratch with a GitHub repo and deployed by Netlify (It takes my now only 60 seconds 🚀). But hopefully, I will not need this skill so frequently in the future. 😊 In addition to being more comfortable with Git and Github, I had learned general strategies to avoid similar problems in the future!

And there also came some encouraging signals from the community: Both — Georg Cushen and Yihui Xie — responded almost immediately to my posts. Even the problem was not solved at that time; I decided to stick with the academic-theme. From some other interactions (issues and questions to the theme), I had built up trust to the theme author. George is committed to his theme and very busy to deliver the best product possible. So after a roll back to an older version, I began with this article. As it stands now, this was the right decision. George and Yihui solved the issue together within two days.

Lesson learned from my own experience

The following list is an attempt to turn my Odyssean experience into constructive advises for other users. I am sure it is not complete and maybe in some points faulty. If you disagree or have other, better tips: please comment on this post!

I am not going into details, and I take for granted some basic knowledge/experience with RStudio / blogdown / Git / Github and Netlify. To have read the blogdown book is another requirement.

How to choose a theme?

Most important: In choosing a theme do not only focus on functionality and pleasing design.

Additionally:

Advice 01: Look if the theme author is currently active. If there are no updates for several months or later, reconsider to choose this theme.

Advice 02: Look at the repo to decide if the author is responsive to reported issues or pull requests.

Advice 03: Look if the author provides releases from time to time. Unexperienced users like me are still not comfortable with Git/Github. Instead of forking and synchronizing repositories, I prefer to install updates via ZIP files.

Advice 04: Look into the documentation to see if the explanations are comprehensive and up-to-date

Advice 05: Investigate the popularity of the theme. This check is not easily done but improves the changes for support through other users. Look not only at the number of stars and forks in the repo because that could be misleading for your purpose and community.

Advice 06: Recherche also if people often write in blogs and forums about the theme and what kind of community is using this theme.

… keep in mind that a more complicated and fancier theme may require you to learn more about all the underlying technologies like the Hugo templating language, HTML, CSS, and JavaScript. (Blogdown Book)

How to start working with a theme?

Most important: Do not begin with adapting the design of the theme.

Instead:

Advice 07: Start a test site with a test repository and test deployment with Netlify.

Advice 08: Copy the content of themes/<your-theme-name>/exampleSite to your project directory. Not the folder itself, but just the content inside (e.g., the folders static, content and the config.toml). Overwrite these files in your project directory — it is only a site for testing.

Advice 09: Read accompanying instructions and go line by line through the config.toml. Adapt this configuration file to your needs. Make notes not only how but also why you did which setting. Documenting these changes could be helpful later when you have already forgotten you initially considerations and the procedures to follow.

Advice 10: Now start writing text in your test site. I recommend working on real posts you want to publish. Choose predominantly use cases which will be typical for your everyday work. This advice seems strange as you are working still on a test site. It is important to test the theme under real working conditions. Besides, writing real posts are not lost work: You are producing standard text files which you can transfer to other sites or themes if the need arises.

Advice 11: Establish a common workflow and get used to it.

I use the following workflow

I start my daily routine with blogdown::serve_site() using the RStudio addins plugin (CTRL-S on my machine) provided by the blogdown package.

Update 2021-05-19: Due to significant changes starting with blogdown version 1.0 my workflow has changed slightly: It is not necessary anymore to use Build -> More -> Clean All because you can render and publish just one page.

Then I start writing. After saving the file (which automatically updates the local website), I look over the result locally in my browser. (I prefer my Google Chrome browser for detailed inspection and use the RStudio viewer pane only for a general overview or to look up a specific change.)
From time to time — or whenever something seems wrong or not updated — I use Build -> More -> Clean All followed by restarting R. And again blogdown::serve_site() to update locally.
After I have finished my work, I commit and pull the changes to my repo, and after some seconds, when Netlify has deployed the site, I inspect the live result.

Advice 12: After some time with your daily working routine, you get accustomed to all the other features of the theme. Even if there are functions you will not need to use: It is nice to have an overview of what could be possible and what not. This knowledge is also important to understand future updates of the theme.

Advice 13: Not until you feel comfortable with the functionality of the theme and your working habit, you should start to adopt the theme. Here it is important to do all your work on your side of the project; meaning not under the themes directory. Copy those files you are going to change or add under the same structure in your home folder of the RStudio project. Changes on these files will overwrite the virgin theme. One the one hand, your changes are all collected in one place and separated from the original theme. On the other hand, new versions of the theme will not override your changes.

Advice 14: Only when you have finally arrived at a status where you work could be publicly launched, change to the real website. Prepare a fresh installation with your theme, with a new repo and a new deployment by Netlify.

Advice 15: Before you copy your work into a new RStudio project, create a new Git branch and check it out (= change to it). If anything goes irreparably wrong (and believe me: eventually in the long term this will be the case!) you have not destroyed your current web presence. After my odyssey, I worked with two branches besides master: config for bigger changes (new version of the theme, changing the configuration or the structure of my site) and blog for my daily (writing) work. If you decide to merge your branch with master you will have an additional security level: Netlify will check if anything went smoothly before your branch is merged with master. If something went wrong, you would abstain from merging your branch.

Problem-solving strategies

Most important: Keep calm and RTFM!

Advice 16: Use Google (or other search engines) to see if other users experienced the same problem. Start your search inquiry with blogdown <name of your theme> <error message> or different appropriate combination of content. Most of the time, you will get results linked to questions in Stack Overflow or blog posts where other users reported about the same or similar problems.

Advice 17: If you cannot find appropriate problem/solution pairs, I would now — after my Odyssean experience — not yet starting to bother people at discussion fora. Start instead of a new reference (test) site with the XMin theme and try to reproduce the problem. If it works there, then the problem has to do with your theme or the changes you have made.

Advice 18: If the problem persists in your theme, then there are still two possibilities: Either you did something wrong, or it really is a bug in one of the pieces in the R Markdown machinery. Produce a REProducible EXample ( aka Reprex). Read Reprex do’s and don’ts and report the problem at Stack Overflow or to the RStudio Community[(https://community.rstudio.com/). Your replication on the otherwise empty reference site will help you to focus on the essential question and will also provide an excellent reproducible example.

Tom Cruise Help GIF from Tomcruise GIFs

Figure 1: Tom Cruise: Help me, help you!

Several updates

This post has enjoyed several updates already!

Back to docdock

Update 2019-06-03: As you can see from the footer, this website is using the docdock- and not the academic theme I wrote above. I started with the docdock and abandoned it for reasons I mentioned above. After another year of creative abstinence of blog writing, in May 2019, I fired up the blogdown machinery again. But because of a combination of many updates of Hugo (from 0.27 to 0.55.6!) and the academic theme (from 2.4 to 4.3), I not only ran into a bunch of error messages, but I have also lost track of different breaking changes and couldn’t recover. —😳

So I changed to docdock back, but this time with somewhat more knowledge. Hopefully, my activities were not a circle but a helix movement where I managed some advances. Thinking positively, hopefully, I can stick with this theme in the future. I do not want writing another complaining article next year, addressing the same problems again.

Not docdock but Pandoc was the problem

Update 2019-06-07: The reason for my problems mentioned above was an old version of pandoc. I had installed “pandoc-1.19.2-29-jan-2017”, but the latest release of the date of this writing is “pandoc-2.7.2-5-april-2019”. Now I could successfully update Hugo to “v0.55.6” By this occasion I also updated jQuery v2.2.3 to v.3.4.1.

Now there wouldn’t be any reason not to return to the academic theme. But I had worked hard the last week and learned so much about my docdock theme, that I am feeling now very comfortable and will stick with it.

Voila! Back to Academic theme again!

Update 2021-05-19: As you can see from the footer, I am back at the Academic theme again!!! The desired stability of the theme development became an unsupportable disadvantage: I couldn’t use the new features of Hugo any more. With some breaking changes (Content organization via Page Bundles and Goldmark as the new default Hugo markdown library) I became with the docdock theme sidelined.

Why Academic (now Wowchemy web page builder) and not a more simple theme?

After four years of learning and (sometimes bad) experiences, I am feeling more comfortable with the whole production chain and have more confidence to master the complex theme now.
Some changes in blogdown (several checking functions and the possibility to fix the Hugo version) addressed the problem of Hugo as a moving target where users had to follow every revision and update.
Both parts of the community (blogdown and Academic) have grown essentially, so help from other users is much more likely.

What is obvious, and for whom?

Tue, 17 Oct 2017 00:00:00 +0000

In the last few weeks, I tried to work with bookdown, an R package developed by Yihui Xie. This program is/was for me much harder to understand, unlike blogdown by the same author. In blogdown, I could even – after some initial problems – write a tutorial which even the developer applauded :sweat_smile:.

But with bookdown, it was different: Not only that I misunderstood some program functions but more importantly, I could not adequately report my problems in one case, I did not communicate vital information in the other case I did not look at the right place where my question was explicitly addressed. It is not a surprise that Yihui Xie got angry with me and even wrote a blog post about the incident. He gave me the advice:

Please do not make these assumptions when filing issues

Be aware: This is a somewhat theoretical/philosophical discussion.

For me, as a professional educator, this is a fundamental question. Where to start with explanations and where to stop? Should I provide just the information to solve the problem, or should I give some background information? What kind of advice should I give to prevent similar issues in the future? These are examples of questions I have to consider as a teacher all the time. But this time, I am on the other side of this communication exchange - I am the learner and student, and I am the one who has to ask the expert.

Yihui Xie is not blaming me as a user, as he wrote in his last paragraph of the mentioned blog post. You can see his positive attitude in the post on The Minimal Reproducible Example Paradox where he is complaining that many users do not provide a minimal reproducible example or do not know how to produce one. He comes to a somewhat depressing conclusion:

How many times have I reminded a user of posting a minimal, self-contained, and reproducible example (reprex)? Probably 500 times. How many times do I think I will still need to remind users of this? Perhaps 5000 times.

But again: This is a typical situation we have in education as well. We are explaining the same thing over and over but — to different people, students, learners, classes, etc. Even if we explicitly mention common errors and misunderstandings, some learners will always commit these mistakes. When we describe operations you should not undertake, you can be sure there are one or two who will follow precisely these wrong paths.

Tacit knowledge and language games

So what? Are people dumb? Do they deliberately ignore advice? I don’t think so, but instead, there is a gap which cannot be overcome only with linguistic means. It is also a question of transferring tacit knowledge’ to use a notion introduced by Michael Polanyi:

I shall reconsider human knowledge by starting from the fact that {{< hl >}}we know more than we can tell{{< /hl >}}.(Polanyi 2009)

Many philosophers (e.g., Søren Kirkegaard, Ludwig Wittgenstein, Gregory Bateson, Susan Langer, Jürgen Habermas, Nelson Goodman, to name a few) have reflected about the limitation of linguistic expressions. Wittgenstein, for example, has pointed out that we cannot explain the meaning of every word or sentence because we have to use language with other words and sentences. It is an endless regress that we cannot solve with language alone. We have to share a common ground, and we have to know how “to play the language game.” Following some quotes to give you a flavor of the argument:

One has already to know (or be able to do) something to be capable of asking a thing’s name. But what does one have to know? (Philosophical Investigations 30) … When one shows someone the king in chess and says: “This is the king,” this does not tell him the use of this piece-unless he already knows the rules of the game up to this last point. (31) … Someone coming into a strange country will sometimes learn the language of the inhabitants from ostensive definitions that they give him; and he will often have to ‘guess’ the meaning of these definitions; and will guess sometimes right, sometimes wrong. (Wittgenstein 1961)

To use a famous philosophical example (Quine 1960): If I point to a white rabbit and say in my foreign language, “Gavagai!” you have to guess the meaning. Do I say, “There is a rabbit!” or “Look at the nice white color!” So even an ostensive definition is under-determined and does not help to choose the right interpretation and to solve the ambiguity of meaning in our speech acts.

One proposal which could serve as an avenue of escape for this gap or paradox is to invent a special kind of notational system. Common examples to demonstrate this strategy is chess notation or musical notation. I believe that a minimal reproductive example or to deliver the sessionInfo() is also a kind of notational system. But the problem is the same: One has to learn this (new) language to use it correctly.

Many different layers

The malice of the situation is the assumption that from the fact that we are using the same (English) words and sentences that we communicate the same meaning and understand to each other. The expert believes that (s)he has expressed every necessary information and so the novice in asking a question. – And this is mostly correct – from his/her point of view or understanding.¹

I have reflected why for me, using bookdown was so much more complicated than blogdown. I started several months ago with bookdown, but I gave up and started another trial after I had some success with blogdown. This sequence seems weird as it appears that for Yihui Xie, the situation was inverse.²

I think there are different reasons:

Blogdown was written together with Amber Thomas and Alison Presmanes Hill. It was an excellent inventive and successful strategy to integrate authors who can provide the perspective of the user side. In the blogdown book, there are some beneficial chapters about action strategies. They give, on the one hand, practical tips and contain, on the other hand, some (technical) simplification. We, educationalists, call this method “pedagogical reduction”). I refer to chapters like A quick example or A recommended workflow.
The success in working with blogdown is more straight and comes in smaller bits and pieces. You write a sentence with Markdown in your blog, and you can see the result immediately. It seems - at least for laypeople like me - that in blogdown, there are not so many hidden conversion processes as this appears to be the case in bookdown. Bookdown, by definition, provides the means for cross-media publishing resulting in three different products (website, PDF, and ePub), all three with their underlying structure and rules.

The importance of mental models

But there is one common point why both blogdown and bookdown are intrinsically difficult to understand for novices: Many different layers of software are working together to produce the result. I am not sure if I am even able to enumerate these different layers correctly: There is

the used hardware,
the operating system,
R with all its various packages,
the programming and writing environment RStudio,
Rmarkdown using pandoc,
knitr,
blogdown or bookdown,
the used theme,
with needs for adoption CSS knowledge,
git and GitHub,
and last but not least, Netlify.

All of these different tools work on top of each other and/or together. And all of these different levels have their complexities, laws, functions, manuals, commands, etc.

As long as everything works fine, there is no need to understand the different parts and their interaction and/or synergy. But when something breaks down, then a shift of focus occurs, or as Heidegger says, the worldliness becomes obvious, changing tool usage from “ready-to-hand” to “present-at-hand”(Heidegger 2008). In “ready-to-hand” awareness, you are using tools to fulfill their purpose (to write a blog post); in a breakdown (= “ready-to-hand” awareness), you get consciousness of the complexities inside your tools machinery.³

But here comes now the problem: As users, we are only experts in “ready-to-hand” awareness, and we have no clue about the functionality inside the black box. We can only report some weird behavior about some experienced phenomenon from the “ready-to-hand” perspective. We, laypersons, have constructed a mental model about the functionality of our tools, and we are reporting about the problems with the underlying implicit assumptions of our mental model. The mental model functions as a world view: We see all the different objects, people, etc., under a particular perspective guided by our underlying tacit assumptions.

After the breakdown, we should question our mental model, but this is very difficult: Our worldview has worked successfully so far. One will not change the world view with just one different fact. The same happens with scientific theories: One other point is a challenge to elaborate on the theory, improve it, and incorporate the anomaly into the argument. Maybe the problem lies in the reluctant fact itself (e.g., the software has a bug, or the author misreported the effect).

Lesson learned

So what the fuss with all these abstract ideas? Where is the practical impact of all these considerations? — I make the case in this article that we should focus separately on two different kinds of explanations:

On the one hand, we users need “ready-to-hand” reasons which are easy to follow even if they are simplified and do not cover all eventualities. We need guidelines, step-by-step (video)-tutorials, etc. Are there shortcuts, tips, and rules of thumb one could follow? And we would need tons of examples! But people who provide this kind of explanation should be aware: This explanation always is under-determined and cannot prevent misunderstandings. Lacking the necessary knowledge, we still have to interpret what “Gavagai!” means.
On the other hand, we users need at the same time “present-to-hand” explanations to build up correct mental models of the inherent complexities and functionalities of the tools. Which software components rely on which other parts? What is the task of component X, and how is it related to the ready-to-hand functionality? But people who provide this kind of explanation should be aware: This level of explication is not a technical one, not directed to other experts. (This is another critical third level of description and documentation, one I will not cover here in this article). So we would need individual formats like diagrams of the interconnectedness and synergistic effects. The result is a kind of multiple representations and would provide additional access to discover the functionality of different parts of the software machinery.⁴
A piece of advice like “Please do not make this assumption when filing issues.” are basic ethical guidelines but not very practical and helpful. We are always using (implicit) assumptions. In stating one proposition, we rely on a set of different assumptions; otherwise, we could not use language and construct meaningful sentences. The problem is to know what assumptions are to test and what assumptions are to rely on.⁵

There is no possibility to examine all belief systems at the same time. The whole book “On Certainty” by Ludwig Wittgenstein Wittgenstein (1975) is full of aphorisms about tacit assumptions we rely on. Our knowledge does not consist of separated propositions but is a system of interconnected views/opinions. I will close this post with some quotes by Wittgenstein to illustrate this point.

§139. Not only rules, but also examples are needed for establishing a practice. Our rules leave loop-holes open, and the practice has to speak for itself."

§142. It is not single axioms that strike me as obvious, it is a system in which consequences and premises give one another mutual support.

§144. The child learns to believe a host of things. I.e. it learns to act according to these beliefs. Bit by bit there forms a system of what is believed, and in that system some things stand unshakeably fast and some are more or less liable to shift. What stands fast does so, not because it is intrinsically obvious or convincing; it is instead held fast by what lies around it.⁶

StackOverflow for instance, is very, very good for “ready-to-hand” explanations but does - as far as I know - a lousy job for building up mental models. Discussions about different but similar software packages, their advantages and disadvantages are forbidden.⁷

References

Heidegger, Martin. 2008. Being and Time. Reprint. New York: Harper Perennial Modern Classics.

Polanyi, Michael. 2009. The Tacit Dimension. University of Chicago Press.

Quine, Willard Van Orman. 1960. Word and Object. The Mit Press.

Wittgenstein, Ludwig. 1961. Philosophical investigations. New York: Macmillan.

———. 1975. On Certainty. Revised ed. Oxford: John Wiley; Sons Ltd.

I could tell some jokes about misunderstandings between Germans and Austrians, both using German as their native language. ↩︎
He said that blogdown was especially complicated to develop, but I cannot find the place at the moment to quote it correctly. ↩︎
For an understandable description of the philosophical concept of ‘breakdown’ read this example on Human-Computer Interaction ↩︎
An essential step for me in understanding the complex template structure of Hugo was to write in every template the sentence: “This here is the template ” and to see what happens in different situations ↩︎
BTW: This is the same conclusion as Yihui Xie has drawn: “To clarify, I’m not blaming this user. I can totally understand it. Many users do the same thing. The problem is that there are so many possibilities for software to screw up, so you have to describe everything clearly to eliminate as many possibilities as possible.”— So we do have the same analysis. Still, I propose (in addition to reflect and report all hidden assumptions) another new long-term strategy (namely helping to build up correct mental models about the complex software dependencies) to prevent similar faults as I have committed. ↩︎
To prevent a possible misunderstanding: I do not think that only or even predominantly software engineers are responsible for the two mentioned explanation levels. Their expertise is to develop programs, document them, and resolve bugs. I just want to point out that there should be more focus on these two levels and that more people should be concerned about providing information on these two levels. ↩︎
At least this is *my* mental model from StackOverflow For instance, the moderator closed my question about the difference between packrat and checkpoint as off-topic. ↩︎

Disqus installation again

Tue, 03 Oct 2017 00:00:00 +0000

A short personal note on my tutorial on Disqus installation: I have used for my screenshots, not this docdock but the hugo-academic theme. I have changed from docdock to hugo-academic and now (2017-05-31) I am back at docdock again. I will explain the reason for this twofold change in one of my next blog post. I am also planning to update text and screenshots to the new blogdown versions.

Update: 2021-05-19: Almost exactly four years later I’m ruefully back at the Academic theme again.🤭

In my tutorial How to install disqus on Hugo I have used the hugo-academic theme to demonstrate the installation process. The article was one of my first contact with static websites, and I experimented with it at https://portfolio.peter-baumgartner.net.

After some experiences with the academic website, I looked for other themes which are using more prominent the blog function. I came up with this beautiful and very functional hugo-theme-docdock website for documentation. But for the tutorial, I have used a different code for the integration of Disqus proposed by Yihui Xie.

In the meanwhile, Yihui has suggested his approach as a pull request which was accepted by the Hugo team. So I assume that in a new release we will all get this more elaborated disqus integration code in the standard Hugo installation.

Alternatives for Disqus?

Thu, 14 Sep 2017 00:00:00 +0000

Disqus is a top-rated service for hosting and managing comments. But it has as an external service several disadvantages which opposed the philosophy of static websites diametrically. I discuss some alternatives for integrating discussion fora with static websites.

Disqus has a long list of drawbacks

In one of my tutorials I have explained how to integrate Disqus into your static website. But in this article, I recommend looking for alternative comment services. Why this different standpoint? In the meanwhile, I read several articles questioning Disqus. Here is a list of critiques I found on the web (without ranking):

Disqus is slow and has a bad (re-)loading behavior.
Disqus is tracking many different things for different customers, some of them hidden and unknown.
Disqus does not allow Markdown.
Disqus does not allow anonymous content (IP address, email, and name are recorded).
Disqus is hosted in the USA, which is considered to have less strict privacy laws than Europe.
Disqus is not open source.
Disqus does not allow that users can use free licenses for their comments. It is not clear who has ownership of the comments.
Disqus is one central authority collecting all comments of your website visitors and users on their servers.

A typical request log with Disqus enabled: There are 105 network requests vs. 16 without Disqus. Load-time rises between 2 to 6 seconds.

Especially the first and last bullet points defeat the advantages of a static website: Speed and all content always at your disposal. On a static website, data are just plain text files, saved locally on your hard disk. Therefore you can static sites transfer easily: Compress all your data in one zip file and unzip it where ever you want it. With Disqus, these advantages are not valid anymore because your blog text and its comments are hosted separately on different servers.

What are alternatives for Disqus?

Summarizing the disadvantages, I mentioned above I am looking for a system which is

free
open source
and does not host the user-generated comments centrally

The Hugo page on comments lists six available alternatives for comment on static websites¹. On AlternativeTo you will find 16 systems².

Cursory review on some services

Staticman

Screenshot of Staticman’s website

Staticman is open source and transforms user-generated content into data files to merge in your GitHub repository, along with the rest of your content. This approach seems promising for me, but until now, I couldn’t manage to install it. I have reported my problem, and I am currently waiting for help. As soon as I know how to fix it, I will review the system here on these pages.³

hyothes.is

Screenshot of hypothe.is' website

hypothes.is wants to provide “a conversation layer over the entire web that works everywhere, without needing implementation by any underlying site”. As far as I understand, for this approach, the data has to be stored centrally. So this software again is not an alternative to Disqus. This remark is not a critique of hypothes.is because it belongs to another category of software services. It has a new approach worth reviewing later in more detail.

IntenseDebate

Screenshot of IntenseDebate’s website

IntenseDebate is a feature-rich comment system for many blogging resp. CMS platforms. IntenseDebate is developed by the people who are behind many other well-known software services (e.g., WordPress.com, WooCommerce, Jetpack, Simplenote, VaultPress, Akismet, Gravatar, to name a few). It seems a bit odd that I could not find newer information on their blog than January 2014⁴. It is free but not open source, but it hosts the content centrally in the US. IntenseDebate is, therefore, no candidate for replacing Disqus.

Graphcomment

Screenshot of Graphcomment’s website

Graphcomment is a beautifully designed commenting service with a (limited) free plan. But it is disqualified under my criteria as the code is not open source and it hosts the comments centrally too.

MUUT

Screenshot of MUUT’s website

MUUT does not allow pre-moderation, e.g. every comment is online immediately. This is an interesting approach, but it is no alternative to Disqus: There is no free plan, and all rights of the user-generated content belong to MUUT!

ISSO

Screenshot of ISSO’s website

ISSO is a lightweight commenting service, programmed in Python, which allows anonymous comments. It is free, open-source, and installed locally. So it does qualify! But the installation procedure seems complex as there is no GUI, and one has to use the terminal for the installation. Furthermore, it seems to me that not all operating systems are covered. But I should give it a try anyway and review it.

Discourse

Screenshot of Discourse’s website

Discourse is a feature-rich open-source environment, supports Markdown and allows anonymous posting. As a hosting service, it has no free plan and costs a minimum of US$ 100 / month (with ~~80%~~ 85% discount for educational resp. 50% for non-profit institutions.). But you can install Discourse yourself without cost on your server. ~~Alternatively, you can pay a one-time fee of US$99 for a cloud installation with a US$10/month hosting fee.~~⁵ With the possibility to install it on your server, Discourse is another candidate to try out.

Using GitHub ()

Screenshot of Gazoo.vrv, Don Williamson’s website

Using GitHub: Another website is also recommending to use GitHub for comments. I have no clear idea how these proposed code lines will work in practice, but if it works, then it will certainly qualify: Open source, free, supporting Markdown and hosted by the website owner.⁶

Summary

I reviewed superficially different commenting systems. I was looking for a free, open source system, allowing anonymous content, Markdown and hosted by the website owner. Four services seem to fulfill my criteria: Staticman, ISSO, Discourse and a code proposal by Don Williamson. hypothes.is is an exciting project, but not a commenting system. It belongs, therefore, to a different category of software (annotation systems).

A competent review of these four systems requires a test installation, which I plan to do in the next few weeks.⁷

Update 2021-05-18: There are now ten services listed. ↩︎
Update 2021-05-18: There are 29 services listed, but not all are open source, and some have already discontinued their service. ↩︎
Update 2021-05-18: Even with some help by the developer I couldn’t manage to integrate it. Seems that my knowledge four years ago wasn’t enough. ↩︎
Update: 2021-05-18: Their blog is not updated anymore, but the main website has recent information, as the footer with copyright 2021 demonstrates. ↩︎
Update 2021-05-18: I adapted the changes of plans. ↩︎
Update: 2021-05-18: Now - four years later - I understand this approach perfectly. There are, in the meanwhile, already a bunch of different services using GitHub. In the new version of this blog, I am using utteranc.es a lightweight comments widget built on GitHub issues. ↩︎
Update 2021-05-18: I have never done these test installations. But at least I have revised this old article. I am still planning to look into details at least of some of these commenting systems. I am using now utteranc.es, and the Academic theme of this blog provides commento, another alternative to Disqus. ↩︎

How to install Disqus on Hugo?

Sun, 10 Sep 2017 00:00:00 +0000

First of all you have to sign up for Disqus.

Create an account on Disqus.

2. Install Disqus

You are offered two choices after the registration process. Select ‘I want to install Disqus on my site’.

Choose ‘Install Disqus’

3. Fill in the form

Disqus needs some information for adding a forum to your website. I will add just two remarks as the form is mostly self-explaining. Firstly, the website name is a text snippet that will appear in the head of every forum section. It does not have to be related to you website name. Secondly, make a note of your Disqus short name as you are going to use it in step 5 and 6.

Configure Disqus for your site

4. Find discussion template

With the next two steps we are going to fix the problem of a faulty Hugo template. Look for the file ‘single.html’ and open it in the editor to inspect the content. single.html should be under /themes/<your-theme-nam>/layouts/_default. At the end of the file – just before the footer –, you should find the name of the partial template for your discussion forum (= ‘comments-html’ in my case).

Configure Disqus for your site.

5. Replace content

Now go and edit this file, which should be found under /themes/<your-theme-nam>/layouts/partials. You have to replace the whole content with the code snippet provided by Yihui Xie (Thanks a lot by the way!!). To help you, you can compare my changes at GitHub. But don’t forget to change the URL to your Disqus account [as I did :blush:]!

Replace content of your partial template as explained

6. Add shortname

To finish the installation you have finally to add the line disqusShortname = "<your-disqus-shortname>" to your config.toml.

Add your Disqus short name into the config.toml

7. Congratulation!

You have integrated Disqus into your static website. Go to one of your posts and check it. Your end of the page should look similar as in the following graphic.

Disqus forum successfully integrated

Critical remark to Disqus

I should mention that there is growing skepticism in the community about Disqus. People are not only criticizing slowness (load-time behavior) but above all they are concerned about data integrity. It seems that there are lots of irrelevant requests to track the data traffic for whatever purpose nobody knows.

Four years later I have changed to utteranc.es.

Blogdown tutorial (Part 4)

Sat, 09 Sep 2017 00:00:00 +0000

Bring your website online: From blogdown via GitHub to Netlify

Update 2021-05-19: Because of breaking changes in Hugo and new features in blogdown this post is partly outdated. Starting with blogdown version 1.0 you do not build the website always from scratch.

Complement this sequence of the tutorial with reading Up & running with blogdown in 2021 by Allison Hill, co-author of the blogdown book.

We start this last part of the tutorial where we have finished last time.

Ignore public folder

One benefit of using Netlify is that that there is no need to upload your large public folder to get your website. Netlify will compile the source files and build by its own the Hugo site. You should take advantage of this feature by telling RStudio that the public-folder does not need version control, committed and uploaded. Select the public folder under the Git tab on the right upper pane and select from the configuration menu ignore….

Ignore public folder

Confirm .gitignore

This will bring up a window where you can specify which files are to be ignored. Confirm the inclusion of the public folder with clicking ‘Save’.

Confirm the exclusion of the public folder.

This saved file (.gitignore) will be added as modified to the version control.

.gitignore file added

Clean up

Before we are going to rebuild the complete site, I recommend to take some precaution. To rebuild from a clean slate I restart R (Under the RStudio menu ‘Session’ or CMD-Shift-F10 on MacOS.) and clean up all unnecessary files (‘Clean All’ in the ‘Build’ tab under the ‘More’ menu in the upper right pane.)

Rebuild website from a clean status.

Build website

Now you can safely (re-)build the website by clicking into ‘Build Website’. You get a message of all the activities done, the time necessary for the build and were the output was created. [Personal comment: Frankly I do not know if the last two steps are really necessary when Netlify is running its own scripts.]

Rendering website content.

Commit changes

The next step is to select all changed files under the Git tab and to choose “Commit”, write a commit message and to confirm these changes.

Select (stage) content for next commit.

Bring all changes under version control and check if everything went smoothly.

Push changes

Now you can transfer all your committed changes to GitHub. After clicking on the push button a window opens and you can see if the changed files are transferred successfully.

Push the committed files to GitHub.

Inspect repo

If you don’t trust this process, you can direct your browser to your GitHub repository and check if really the last commit “First post” has safely arrived.

Inspect your repo on GitHub to confirm the procedure.

Get Netlify account

The last task we are going to solve is the connection the GitHub repository to the Netlifly service. Go to https://www.netlify.com/ and create a free new account (“Get started for free”).

Inspect your repo on GitHub to confirm the procedure.

Let Netlify access your repo

After the registration procedure you get a page where you can choose ‘New Site from Git’.

New Site from Git

There are essential three step to create the new website: (A) Choose GitHub as your Git repository. This will bring up a window where you have to confirm that you will allow Netlify access to your GitHub account.

Autorize Netlify

(B) Then you have to specify to which repository Netlify should connect.

Choose your website repository.

Deploy settings

In the third and last step (C in my former notation) on Netlify to create your website you have to specify the details (parameters) to let Netlify know which type of website it has to build. In the example in the screenshot I said that Netlify should watch my master branch, build a Hugo website from the public repository with the newest version of Hugo at the time of writing this tutorial (version 0.26).

Specify the deploy settings.

Deploy website

After clicking “Deploy website” you can lean back and watch Netlify busy working on building the website. This may take for this initial step some time but later on Netlify will finish work almost immediately as it only builds the new/changed files incrementally. When the rolling down of the messages stops you should see as last line: “Site is live”!

Wait — and watch Netlify working until it is finished.

Preview deploy

Scroll back to the beginning of the log messages. You will see that Netlify has giving you a random URL but you can change this later on to your liking. Click “Preview deploy” to go to the URL of your new website.

Inspect your random URL.

Website is online!

Congratulations! Your website is now online.

Your website is online.

Final comments

This was a very quick tour where I have covered just the things I believed to be essential. If you are (like me) new to static website generators then the different steps together seem overwhelmingly complex. But with patience and some trial and errors you should succeed — as I have. There are many other (better) tutorials and How-Tos around. You should definitely read carefully the book blogdown: Creating websites with R Markdown and the reports of different experiences from the new rbind community.

Blogdown tutorial (Part 3)

Fri, 08 Sep 2017 00:00:00 +0000

Live preview of your website

Update 2021-05-19: Because of breaking changes in Hugo and new features in blogdown this post is partly outdated.

Complement this sequence of the tutorial with reading Up & running with blogdown in 2021 by Allison Hill, co-author of the blogdown book. With some knowledge the explanation of the changes of blogdown version 1.0 could also be useful.

In this third part of the blogdown tutorial I will explain the necessary steps to get a live preview of your website. The main advantage to bring up your website locally is that you can view and test your website during working on your post without the need of an internet connection. You have not even to reload the page as this is done automatically through a technology called LiveReload.

Warning after installation

After we have finishing the installation we have gotten a red warning. It is a reminder not to transfer files which are not used displaying the website. At that time we didn’t bother about it. I have replicated the picture of step number 8 of the first part of this tutorial as a starting point for this third part of the tutorial.

RStudio’s four pane view immediately after installation.

Ignore some file types

After the installation config.toml is opened automatically by RStudio. We add the recommended line ignoreFiles = ["\\.Rmd$", "\\.Rmarkdown$", "_files$", "_cache$"] in an upper place. On this occasion we also change the base URL (first line) to /. After saving theses changes the version control (Git tab right upper pane) will display config.toml with a blue M (modified).

Some changes in the config.toml

Demonize server

Before we start posting our website locally we should arrange that the local server runs as a background process (= aka to demonize the server). You can run the command options(servr.daemon = TRUE) on the console (lower left pane), but – as I am using this process all the time – I prefer to write this line into my start up file. On this occasion I also add some other convenient commands to facilitate the generation of new posts. See the blogdown page on global options for more details.

Set Options in Rprofile.

Start local server

We can now start the local servers: Choose from the “Addins” menu the “Serve Site” option. This will run the command blogdown::serve_site() to build the site to get a live preview of the website locally.

Choose ‘Serve Site’ to get a live preview of the website locally.

Building the website

Blogdown is building the site and will give you the local server address. It is normally http://127.0.0.1:4321/. While the server is demonized you can still use the console, otherwise a small red stop sign will appear in the right corner of left lower pane and no command can be run in the console window. The best way to stop the server is to restart R (shortcut = CMD + SHIFT + F10).

Set Options in Rprofile.

First preview locally

We can now inspect our website in a browser with the web address http://127.0.0.1:4321/. Whenever we type something in one of our files and save the changes blogdown will rebuild the site and we can immediately see the changes in the browser via the local address.

First view of live preview locally.

Adapt personal information

Until now all the information this tutorial has provided was the same whatever theme you are going to choose from the Hugo theme wsebsite. But from here now comes your specified theme into play we will look into some necessary adaptions before we going to publish our new website. You can now adapt the personal information in your config.toml file. Read the documentation which comes included as posts of the sample site http://127.0.0.1/#post or go directly to the demo site. I have for instance changed the title of the website from ‘Academic’ to ‘Blogdown Test’.

Changed title from ‘Academic’ to ‘Blogdown Test’

Create first post

To write posts choose ‘New Post’ from the Addins menu.

Create a new post.

Add metadata

This will open up a window to fill in metadata. This form is already populated with the variable from our .Rprofile, so that you only have to add the post title. File name and slug will be generated automatically from the post title. Categories and tags can be filled now — or later written in the file header or via the Addins menu ‘Update Metadata’ added.

Fill in the prepopulated post form.

Start editing text

The ‘New Post’ form creates a new file for the post. The first few lines are filled with metadata of the post. You start with the post text under the second three dashes of the post header. To format your text nicely learn the Markdown

First post entry.

Inspect post locally

After saving your text, you can finally inspect your first post locally! Posts in the hugo-academic theme can be found in post section of the website.

Your first post!

It may be the case that the generation of you first post produces a warning. After Hugo version 0.24 a prototype for new generated files is necessary. In that case create a file with the command file.edit("archetype/default.md") and add the example text written in the warning message.

A warning informs you to create a file with a specified text.

Finally: Visit the final part of this tutorial to bring you website online.

Blogdown tutorial (Part 2)

Tue, 05 Sep 2017 00:00:00 +0000

Creating a Github repository

Update 2021-05-19: Even if some screenshots might have changed slightly, this part of the tutorial is still up to date. But there are now many other resources with the same content available.

Especially I would like to recommend Up & running with blogdown in 2021 by Allison Hill, co-author of the blogdown book. There is a small part at the end to the integration into GitHub.

R User now can use the helpful package usethis with information how to get Git(Hub) Credentials and with Happy Git and GitHub for the useR there is also a gentle introduction to Git & GitHub.

In part 1 we have installed the hugo-academic theme on top of R, RStudio and blogdown. We will continue the installation process with creating a remote GitHub repository.

Create a GitHub repo

In case you do not have a GitHub account, sign up for it. When you have finished the registration process click the button “Start a project”. If you already have an account click at “New repository”.

Creating a new GitHub repository.

Name your GitHub repo

Both actions (“Start a project” and “New repository”) opens up a window where you can name your new repository. Leave all the other options empty resp. as they are and click the green button “Create repository”.

Name your new GitHub repository.

Quick Setup

The last click brings you to a “Quick setup” page with several options. As there already exist a repository with our hugo-academic files, we copy the two lines in the third option: “push an existing repository from the command line”.

Quick setup: push an exiting repository from the command line.

Shell commands: git init

But before we can paste these two lines into the terminal in order to link our local with the remote GitHub repository we need to download Git and install it. The best description I know of the installation procedure and its integration with RStudio is by Hadley Wickham in chapter 13 of his book about R Packages. There is a online version available. Read the two sections Initial set up and Create a local Git repository. For the mentioned shell commands you can use the Terminal tab of RStudio in the left lower pane.

Use for the shell command the new terminal tab of RStudio.

Git tab in RStudio

5After the shell command git init the local repository is under version control. After you have restarted RStudio you will now see new tabs in the RStudio panes. Click in the Git tab in the upper right pane. This reveals a bunch of folder and some files. The yellow boxes with the question marks under the status column indicate that these file are not yet under version control.

Folders and files which are not under version control.

Add files under version control

To bring these folders and files under version control you have to select all of them and click in one of the check boxes under the column “Staged”. You will see now a lot more files as all the files under the still untracked folders are explicitly shown. The A indicates that these files are new (are just added) to the repository.

New (Added) files to be set under version control.

Initial commit

A click in the Commit tab opens up a new window where you must write a message. This is necessary so that all collaborators (including the future-you!) remembers what was the change about. This first (e.g.initial) commit will bring all the files under version control.

The initial commit brings many files under version control.

Success message

After sending your message by clicking the Commit button you will see a list of all the committed files which are now in the local repository under version control. You can close this window and we can finally add our two lines copied from the GitHub page.

Files are successfully committed.

Connect local with remote repo

Insert the two copied lines from the GitHub repository into the Terminal tab (red arrow). The first line will establish the connection between the local and the remote repository. The second line will push all the committed local files to the GitHub repository.Depending of you internet connection and how many data has to be transferred this can take a couple of seconds (blue arrow).

Connecting local Git with remote GitHub repository and pushing the initial commit.

After this second step we are going to to get a live preview of your website.

Blogdown tutorial (Part 1)

Mon, 04 Sep 2017 00:00:00 +0000

Update 2021-05-19: Even if some screenshots might have changed slightly, this part of the tutorial is – with one exception (header 4) – still up to date. But there are now many other resources with the same content available.

Especially I would like to recommend Up & running with blogdown in 2021 by Allison Hill, co-author of the blogdown book.

In this tutorial I will show how to install the R package blogdown with the help of the new version of RStudio and how to link your local blogdown-directory to a remote repository on GitHub. But before we actually start with the installation procedure let us define the different ingredients (tools) we are going to use:

Tools we are going to use

Blogdown is an R package for creating static websites with R Markdown.
R is an integrated suite of open source software facilities for statistical computing and includes tools for data manipulation, calculation and graphical display.
RStudio is an integrated development environment (IDE) which sits on top of R and facilitates the use of R tremendously.
R Markdown is a file format for making dynamic documents with R. It is based on Markdown (a lightweight markup language with plain text formatting syntax), but it also can contain chunks of embedded R code.
Hugo is a popular open-source static website generator.
hugo-academic is a theme designed for Hugo to create an academic or personal website. Besides special academic features like sections for publications, projects, teaching it also includes a blog section and supports multilingual usage.
GitHub is an Internet hosting service for distributed version control repositories. Is is mainly used for source code management, but it works also with plain text markup languages. GitHub offers all the functionality of Git as well adding its own features.
Git is a distributed version control system for tracking changes in computer files and coordinating work on those files among multiple people.

All these tools are open source and free available.

Preliminary preparations: Installing R and RStudio

There are many tutorials to install R and RStudio. See for instance the video Installing R and RStudio. But at the moment the new version of RStudio which facilitates the installation procedure of blogdown is still not released as standard version. But you can use all these new features by downloading the preview release. As these features will be soon the standard version of RStudio I will explain the installation procedure using this upcoming version.¹

Create a blogdown website

Create a new project

After installing RStudio version greater than v.1.1.28 open the project menu and choose “New Project”.

Create a new project

Create a new directory

A new window opens up: We are going to create a new local directory for our static blogdown website. Choose “New directory”.

Create a new directory

Create a website using blogdown

The selection in the following window is self-explaining: “Website using blogdown”. Here it is the last line in the window. But in the future there may added other project types, so that you have to scroll down to see the blogdown-choice.

Create a website using blogdown

Create a website using Hugo and blogdown

Update 2021-05-19: The Academic theme is now modularized into different specialized templates and integrated into the development of the web builder software wowchemy. To get the complete Academic theme, you have now to write into the Hugo theme field “wowchemy/starter-academic” (instead of “gcushen/hugo-academic”).

Lastly we arrive at the final window we have to answer. Here you can choose the name and location of your local directory which will eventually collect all files of your static website. I recommend to leave all check boxes ticked, as this default value will give you the most support to generate the new website. The program for the static website generator Hugo will be installed automatically as well as sample blog posts will be added. Especially important is the possibility to look into the example site of the theme, because this will provide you with clues about the functionality of the theme.

Create a website using Hugo and blogdown

Choose theme

I have chosen the hugo-academic theme. You can pick your own preferred webdesign via the Hugo Themes page.

Huge variety of Hugo themes to choose from.

Choose Academic (now Wowchemy)

In the “Hugo theme” field you have to insert the URL of the GitHub repository of the theme you would like to use. You will see the address when you click at the thumbnail of the theme and hover your cursor over the “Download”-button.

hugo-academic is as special theme for personal websites in academia.

GitHub repor of Academic theme

Clicking on the download button will bring up the developer’s repository. This is useful even for beginner as you can have a look of the ongoing discussion under the “issue” tab or to ask your own questions. Experienced users can fork the repository in order to adapt the functionality of the theme and/or to suggest code changes to the developer via the push mechanism.

GitHub code repository of the hugo-academic theme.

Create project

When you finally click the “Create Project” button you have to wait few seconds until the selected theme is downloaded and Hugo installed. After the installation is finished RStudio opens up a four pane view. We will go more into the details of these different windows in the third part of this tutorial.

RStudio’s four pane view immediately after installation.

Summary

We have successfully installed the hugo-academic theme in a version controlled local directory. Essentially we could now start to personalize the website and/or write articles resp. posts. But we will continue the installation process with creating a remote GitHub repository which has to be linked to the just created local repository. The remote repository not only operates as backup and as distributed version control (allowing collaboration) but also enables – beside a manual transfer via ftp – additional ways of publishing your website. We will cover these possibilities in the following parts of this tutorial.

Go and visit the second part Creating a GitHub Repository of this tutorial.

To use features supporting blogdown websites the RStudio version has to be higher than v1.1.28: As today (2017-09-04) the actual preview version is v1.1.353. ↩︎

Posts | Thought splinters

Introduction to R for Social Scientists (I2RSS)

Acclamations for I2RSS

Authors experienced in political, social & dats science

Promoting the modern tidyverse approach

Real data, results discussed and gentle dialog with reader

Some drawbacks and failures

Educational inconsistencies

Not a role model for Open Science

repairData - My First R Package at CRAN

R Packages book as a valuable assistance

Goal of the datasets

Five data providers

Repair Café International

anstiftung

Fixit Clinic

Repair Café Wales

The Restart Project

Aim of repairData package

Using GitHub as a Mentoring Tool

Beginner, Intermediate, and Expert Level

Project for Motivation Necessary

Looking for a Human Mentor

GitHub as a Mentoring Tool

An instructive example

Lesson learned

Looking Experts over their Shoulders

Summary

Using Emojis Revisited

Summary

H5P and Introduction to Modern Statistics

How to integrate H5P exercises into statistics?

Narrative Part

Examples

Guided Practices

Test: Fenced Divs

Memento Time Travel

Research with archived web data

How does the Memento API work?

Where to find the desired information?

Contributed Packages

Packages By Name

Packages By Date of Publication

Which page to scrape?

Contributed Packages

Packages By Name

Packages by Date of Publication

Memento protocol in action

Package installation

Mementos Link Types

Memento Craw List

Harvesting Web Pages

Tidy data

Get number of available packages

Visualizing the results

Summary

Images in Different File Formats

Image handling in .Rmd, .md and .Rmarkdown files

Observed results

Additional tip

Images in Rmarkdown Files

Using R chunk with knitr

Example

Code in R Chunk

Code in HTML

CSS

Summary

Markdown via Addins ‘Insert Image’

Example

Code in markdown

Code in HTML

CSS

Summary

Visual R Markdown via Menu ‘Insert …’

Example

Code in markdown

Code in HTML

CSS

Summary

Hugo figure shortcut

Using R chunk with `knitr`

Using r chunk with `knitr`