This analysis is based on the population-based study “Health Related Beliefs and Healthcare Experiences in Germany” (HeReCa) [27–30]. The aim of this study is to assess perceptions and experiences regarding health-related topics. Participants receive an email invitation to complete a short online survey (each 10–15 min) three to four times per year. The Ethics Committee of the Medical Faculty of the Martin Luther University Halle-Wittenberg, Germany (No. 2019-044) approved the study, which was conducted in accordance with the Declaration of Helsinki.

We randomly selected 14–15 municipalities (over 5,000 inhabitants) or cities in each of five federal states (Saxony-Anhalt, Berlin, Schleswig-Holstein, Nordrhein-Westfalen, and Baden-Wuerttemberg) in Germany, and asked the relevant resident offices for a random sample of a defined number of persons aged 18–79 (a total of 10,000 per federal state). The municipalities and cities were stratified to represent the distribution of the population density (dense, medium, or sparse) in the given federal state. We mailed invitation letters to the selected residents (Figure 1). These letters contained information about HeReCa and a web link for registration. In order to finalize the registration process, it was crucial to provide the unique code specified in the accompanying letter and an email address. This email address was utilized for subsequent invitations to complete questionnaires, which were accessible via a provided web link. We sent a postal reminder six to twelve months (depending on the federal state) after the first postal invitation to those who had not yet registered and contacted us with a request to be actively removed from the study. Participants provided their informed consent online and did not receive any kind of allowance.

By March 2023, we had administered twelve different questionnaires in HeReCa. A multi-professional team of health scientists developed questionnaires covering contemporary health issues (e.g., transplantation law in Germany, knowledge about medical rehabilitation, various aspects of COVID-19, and perceptions related to climate change). Participants provided sociodemographic information as part of the first questionnaire. Given that participants were recruited from different federal states over time, some questionnaires were administered in fewer federal states. Starting in April 2021 all participants received the same questionnaires. After an invitation to a specific questionnaire, participants received two reminder emails, one after 2 weeks and another after 1 week, in case they had not completed the survey by then. The online questionnaires remained active for 2–3 weeks after the second reminder email.

The response was calculated as the ratio of the number of respondents to the number of persons invited to join the study (at baseline) or the number of respondents to a specific questionnaire among those invited to participate in the questionnaire (at follow-up).

We calculated the amount of money necessary to recruit the participants. We included the costs of materials (e.g., paper, flyers) and stamps used for mailings, and divided the total cost of the initial and reminder letters by the number of registrations received after the first invitation and reminder, respectively. Staffing and administrative costs, such as fees charged by resident offices, were not included.

We compared the sociodemographic characteristics of participants recruited by the first invitation and the reminder letter using absolute and relative frequencies with 95% confidence intervals (CI) and means with standard deviation (SD). We also compared the content of the responses between these two groups using Chi-Squared tests, accounting for the false discovery rate (FDR). Finally, we assessed the probability of completing subsequent questionnaires in both groups. R software (Version 4.1.1) was used for all statistical analyses.

We sent out 50,045 postal invitations and received 2,216 registrations resulting in a response rate of 4.4% (95%CI: 4.3%–4.6%). These participants will be referred to as “first-attempt respondents” in the following analyses. In total, 45,877 subjects did not reply to the initial invitation within six to twelve months and were reminded about the registration with a follow-up postal letter. This way we recruited an additional 1,130 participants (2.5%, 95%CI: 2.3%–2.6% of those who received a reminder letter; 2.3%, 95%CI: 2.1%–2.4% of those who were initially invited). These participants will be referred to as “second-attempt respondents” in the following analyses (Figure 1). The two groups differed only slightly in sociodemographic characteristics (Table 1). Incomplete responses (missing category) were more common among “second-attempt respondents.” A comparison between the vocational training qualification data of the HeReCa panel and the micro census data in Germany showed a trend toward higher education attainment among HeReCa participants (Supplementary Table S1). We sent the first postal invitations in two federal states (Saxony-Anhalt, Berlin) before the COVID-19 pandemic (November 2019) and in the remaining three federal states (Schleswig-Holstein, Nordrhein-Westfalen, and Baden-Wuerttemberg) during the early phase of the pandemic (between March and May 2020). There was no indication of a higher response during the pandemic (Supplementary Table S2).

Given the overall low response, the main cost component was postal letters. The cost per letter was approximately 0.83€ including printing of materials, resulting in recruitment costs of 19€ per first-attempt responder and 34€ per second-attempt responder (+79%).

We analyzed the answers to a questionnaire containing 81 questions about medical care during and before the COVID-19 pandemic. After adjusting for multiple tests using the False Discovery Rate, only one of the items in the questionnaire showed significant differences between the two groups. This question asked whether the participant had any further conditions that required continuous medical treatment, medication use, or a stay in a hospital in the year 2019. Second-attempt respondents showed reduced odds for this condition compared to first-attempt respondents (Odds Ratio = 0.7, 95%CI: 0.5–0.9). The preceding question in the questionnaire listed common diseases (e.g., hypertension, cardiovascular disease) and the participants could indicate whether they had a specific disease. This question was answered similarly in both groups suggesting that the only difference may be random rather than systematic.

We selected the second, third, and fourth individual questionnaires for each participant in order to analyze how many participants stayed on the panel over time compared to the number of initial registrations. First, we studied the active withdrawal of consent (Table 2). The loss of participants due to active withdrawal was similar after each subsequent questionnaire and was less than 1 percentage point for the first respondents and approximately 2–3 percentage points for the second-attempt respondents per questionnaire.

In contrast, attrition by simply not responding to subsequent questionnaires was much more common among second-attempt respondents compared to first-attempt respondents (Figure 2). After the initial decline from baseline to the first follow-up questionnaire, first-attempt respondents displayed only a slight decline in participation. In contrast, second-attempt respondents showed a more pronounced decline in participation in subsequent questionnaires.

The strength of our study is the population-based sampling across five federal states in Germany. Furthermore, in contrast to studies that sent reminders after a short time period, where respondents to the different waves might overlap, we could fully separate the groups that responded to the initial invitation and the reminder. There are also limitations. Our study was conducted only in German, so it is limited to the native population and well-integrated immigrants (in terms of language). The content of our study is general perceptions and considerations related to health topics, so our findings may not be generalizable to investigations on different topics. The topics of the respective questionnaires were announced in the invitation email to complete the questionnaires and topic preferences could affect the decision to fill out a specific questionnaire.

A postal reminder increased the number of subjects recruited, but those recruited with more effort did not differ substantially in sociodemographic terms from first-attempt respondents. Moreover, first- and second-attempt respondents provided similar responses to the questionnaires presented. Given the higher recruitment costs and higher attrition when using reminders, our results support the use of a single invitation letter in a population-based online study, when participants need to be invited by regular mail.