Between May 17, 2017 and June 10, 2017, we used advertisements to enroll GBMSM from two venues—one of the most popular geo_targeted sexual networking apps for GBMSM and one of the most popular social networking websites for the general population. The sexual networking app pushed the advertisement as a message to the chat inboxes of all users in the United States on Friday, May 19, 2017, which remained for 7 days, unless deleted sooner. On the social networking site, we used _targeted banner advertisements for approximately 4 weeks that could show up in one of the two ways—a static ad on the right-hand pane of the website or an ad that resembled a normal post as users scrolled through their feeds. We _targeted the social networking site ads to people who were men, residing in the United States, aged ≥18 years, and believed to be GBMSM based on either a same-sex interest listed on their profile or a range of relevant “likes” (eg, gay pride, lesbian, gay, bisexual, and transgender, LGBT, community, gay bar, and same-sex marriage). Both ads comprised a background image (the social networking site: 2 clothed men on a bed kissing; the sexual networking app, 2 bare torsos embracing) and brief text, including that they could “enter to win a $50 Amazon.com gift card” and that there was “no participation necessary” to enter the random drawing.

Upon clicking on the ad, the participants were informed that the survey would take approximately 10-15 min to complete and provided the option to begin immediately or enter their email address to receive a link to complete later. Upon beginning the survey—whether immediately or through the emailed link—participants were provided with a brief online consent form and given the options of providing consent, declining, or declining with the option to receive instructions for entering the random gift card drawing. During consent, participants were informed about a 1 in 100 chance of receiving a $50 gift card. Those who subsequently declined consent were provided instructions should they want to enter the drawing; conversely, participants who completed the survey and were interested in entering the drawing were redirected to a separate survey in which they were required to enter an email address that was not linked in any way to their data. During the first few questions of the survey, participants were screened for eligibility, which was defined as follows: (1) age ≥18 years; (2) residing in the United States; (3) having had same-sex sexual activity within the past year; and (4) identifying as male (including both cisgender and transgender males). Those who were ineligible were informed after the first few questions, and the survey subsequently ended. The study protocol was reviewed and approved by the Human Research Protections Program of The City University of New York (New York, NY, USA).

We followed a protocol based on standards within the literature [41] for removing potentially duplicate cases while erring on the side of keeping rather than removing data in cases where a determination could not be made. In particular, we first identified potential duplicates based on birth month and year, zip code, HIV status, and race/ethnicity; all cases sharing those features in common were manually examined, focusing on responses to other questions such as education, employment, and partner status, as well as device and browser information and the survey duration.

We collected all measures for this study as self-reported items and scales within the one-time online survey. The item content was developed in part by consulting the terms of service and privacy policy for two social networking (ie, Facebook and Facebook Messenger) and two sexual networking (ie, Grindr and Scruff) apps in late 2016. In addition, we examined the types of personal information and data discussed within those agreements and the usage provided for within the agreements to develop three primary data activities described in the measure below (ie, data collection, anonymous sale of data, and anonymous sharing of data). Likewise, we used the sites and apps to create a list of the types of personal information (ie, data) that are likely to be gathered and/or generated by developers. After obtaining the complete draft of the measures, we invited a group of 20 adult GBMSM in the New York City area to participate in an in-person community feedback session; all participants were provided with a copy of the measures, and we reviewed both the study procedures (eg, recruitment and compensation) and the item content with them to gather their feedback. We received and followed numerous suggestions to improve clarity, reduce length, and minimize burden. For example, from a list of at least 15 different types of data, community members noted that they were not all meaningfully distinct; thus, the list was condensed to form broader categories in some cases. Similarly, we implemented suggestions for improved wording. The final version of the measures was based on this feedback and a review by field experts from Fordham University’s HIV Prevention and Substance Use Research Ethics Training Institute (New York, NY), as described later (see the Online Supplementary Material for more details).

All analyses were performed in SPSS 24 (IBM Corporation; Amonk, New York, United States). To inform future online recruitment efforts, we began our analyses by characterizing the sociodemographic characteristics of the sample and comparing them across the two recruitment venues using chi-square tests of independence. To address the first aim regarding the comparisons of trust for guarding personal information across nine different sources, we iteratively conducted a series of 36 repeated measures analysis of variance (RMANOVA) models examining each pair of ratings while adjusting for relevant between-person characteristics (ie, recruitment source, race, HIV status, and age); we specified an interaction for each between-person factor with the within-person factor but not among the between-person factors. We reported the η_partial² effect sizes for the within-person main effect as evidence of the magnitude of each comparison. To address the second aim regarding privacy concerns raised about 12 different types of app-related data across 3 different data activities (ie, the app collecting the data, the app anonymously selling the data, and the app anonymously sharing the data with researchers), we assessed the prevalence of indicating each was a concern by examining the frequency and proportion of “yes” responses across the 36 dichotomous indicators. We also calculated a sum score for the total number of types of data that raised concerns for participants for each of the 3 data activities and compared the 3 sum scores to one another in an RMANOVA that was consistent with the prior set of analyses with two exceptions—all 3 scores were compared simultaneously rather than in pairs, and we used a simple contrast to test differences between the three, using sharing with researchers as the referent group. Finally, to address the third aim regarding which types of app-related data participants would hypothetically be willing to provide explicit permission to have shared with researchers, we used the same 12 types of data asked about in the second aim and used a series of 66 pairwise RMANOVAs consistent with the first set of analyses to compare within-person differences among the 12 ratings adjusted for the relevant between-person factors.

Across all analyses, the primary goal was to examine patterns in the data descriptively using effect sizes rather than search for statistical significance, particularly because of the large sample size. Furthermore, we reported the η_partial² effect size as small (0.01), medium/moderate (0.06), and large (0.14) in size [42]. Nonetheless, for statistical comparisons, we reported statistical significance for those findings that reached a threshold of P<.001 to reduce the likelihood of type II error because of multiple comparisons.

We conducted an experimental manipulation to test whether providing a rationale for each of the 3 activities within the “Concerns about privacy threats ” measure would influence trust. Specifically, participants were randomized to receive either a description of the 3 activities with no rationale or the same description with rationale added (eg, for the app owners collecting the information, rationale added was “to improve, tailor, and develop the services you use”). As results suggested nonsignificant and extremely small (Cohen d<.05) differences between groups, all results are presented irrespective of the experimental condition.

We analyzed data from an online sample of 11,032 GBMSM across the United States to examine participant perspectives on the issues of trust, privacy, and data sharing in online and mobile research. In analyses that were adjusted for relevant between-person differences (including the recruitment site), we found that trust in online research was greater than trust in online and mobile platforms for personal use, such as social and sexual networking apps or various types of websites. When focusing on 12 different types of data that could be gathered by a hypothetical sexual networking app, participants expressed the least concerns about privacy when such data were going to be shared anonymously with researchers and the most concern when these data were going to be sold anonymously to third parties; the actual collection of the data by the app owners raised an intermediate level of concern. Finally, reviewing the same 12 types of data, we examined which types of data participants would be willing to share within future research studies—participants were most willing to share information they disclose publicly within the app (such as profile information on characteristics like age and height) and least willing to share information that could be collected by the app automatically (such as GPS location or device usage information).

We found overall moderate levels of trust within online research studies, with little difference based on the type of organization conducting the research. In this study, approximately two-thirds of GBMSM trusted or highly trusted online and mobile research compared with one-quarter who trusted GBMSM-specific networking apps and approximately 18% who trusted networking apps used by the general public. Although not asked in exactly the same way, these findings suggested lower levels of trust in this sample than those in a previous Pew poll [37], which could be due to the population or due to more general shifts that occurred in the year that passed between the 2 studies. Although we asked about trust in online research across three different types of organizations (ie, LGBT community center, a university, and a government agency), participants in this study did not appear to differentiate between online research done by these three different groups and reported similar levels of trust for each. Particularly for researchers to understand, participants expressed greater trust in research than in many of the online and mobile technology companies and services they use on a regular—if not daily—basis. With the proliferation of public-private partnerships and collaborations between research organizations and these service providers, it is critical to consider how this might affect trust within both sources. Efforts to maintain trust by promoting transparency in research practices within such partnerships might prove critical. For example, getting informed consent before having data shared anonymously could be the best practice, even when such permission has already been granted within the terms of service for the app or site and research activities might qualify for a waiver of informed consent based on the federal criteria for human subjects review exemptions if data are transmitted anonymously.

In this study, participants expressed concern about several data collection, selling, and sharing activities. These findings are consistent with a study on the privacy paradox [43], which suggests that individuals’ concerns about privacy are discrepant with their own privacy practices (eg, privacy settings). In this case, the paradox results from participants expressing concerns about the very types of data collection, selling, and sharing that they have likely already agreed to within the terms of service and privacy policies of the very apps and sites they regularly use and from which they were enrolled. Also, somewhat unexpectedly, more participants expressed concern about the actual collection of these various types of data by the app owners than they did about the anonymous sharing of the same data with researchers (an act that would be impossible without the apps first collecting these data). One potential explanation for this set of findings could be that the data remain connected to participants’ identities for the app owners, whereas they were specifically referenced as anonymous when sharing with researchers. Another possibility is that this higher willingness to share data with researchers than have it collected in the first place by the app owners is due to the higher levels of trust in research that were observed within the analyses for the first aim of this study. Nevertheless, further investigation is warranted to explore the potential mediating roles of anonymity and trust on these differential privacy concerns. For example, privacy concerns might be lower for anonymous data activities than for identified ones—people may have similar or even higher levels of privacy concerns about sharing with researchers as they do about the app owners collecting the data if the sharing is not anonymous. Relatedly, people who trust different sources more might also express fewer privacy concerns, and so differences observed may be due to greater trust in research than the technology companies themselves.

Not surprisingly, similar types of data that participants expressed privacy concerns about were those that they were least willing to share with researchers. This might have implications for policies around broad consent for data sharing, whereby participants might need to be given the choice to opt in or out of specific types of data collection and sharing activities rather than simply consenting to share or not share all data. Specifically, these findings suggest that if individuals are given a choice of sharing all data or none, many might select to not share, resulting in low enrollment and high rates of missing data thus biasing the sample and study results. Alternatively, providing options about what to share might, at the very least, allow a more representative sample on some of the types of data (eg, sociodemographics) and could allow for a better estimate of how biased the results are for the types of data not shared. However, this study did not examine the impact of compensation, and further research is needed to examine how compensation might alter participants’ willingness to engage in data sharing; understanding the impact of compensation on data sharing—particularly types of data that participants are otherwise generally unwilling to share—may inform ethical considerations.

Finally, data for this study were collected prior to the recent concerns about data-related and privacy issues on both social networking sites and sexual networking apps [44,45], and replication of the findings in the wake of ongoing privacy-related events is warranted. Future research can and should attempt to understand the magnitude and longevity of the impact of these events on constructs such as trust, privacy concerns, and willingness to share data. In the wake of such events, many technology companies seek to update their privacy policies but may do so with little information on what types of protections are most important to their users—researchers studying privacy in online and mobile technologies both within and outside of research are well-suited to understand and subsequently advise on exactly these types of issues.

In this study, we considered the use of technology and limited interaction procedures as strength as it facilitated large-scale data collection of individuals with substantially fewer resources than would be possible in a standard research study. However, it also necessitated conducting a very brief survey with a limited number of measures. We used a _targeted advertisement with a random chance for incentives along with rigorously implementing standards for confirming the veracity and uniqueness of participants to reduce the likelihood of false and duplicate participants [41]. However, in online studies such as this, some degree of duplication or invalid response is likely. We reviewed the terms of service and other policies for several existing social media and sexual networking sites and apps while developing our measures to contextualize them appropriately. Nonetheless, the study constraints limited our ability to ask about the extent to which participants have ever read these policies, and the extent to which they realize that many of the data types and collection activities assessed were those that they have generally agreed to for apps and sites they regularly use remains unclear. We adjusted for rather than focusing on the role of sociodemographic and behavioral factors and future research is warranted to explore how trust, privacy, and willingness to share data might differ according to factors such as HIV status and race/ethnicity. Finally, conclusions regarding trust in this study and concerns about privacy are slightly limited as this is, by definition, a sample that agreed to participate in an online research study. However, this is also a sample using the apps and sites from which they were enrolled, which were still trusted less than research, suggesting this finding regarding the relative trust could be reliable more generally even if the actual levels of trust are skewed higher by the nature of the sample.

This study suggests a relatively favorable view of online and mobile research—this large sample of GBMSM across the United States expressed a moderate level of trust in online research and few data-related privacy concerns. Moreover, the sample was nearly evenly split based on their willingness to have several types of app-based data shared with researchers, suggesting the analysis of such data might be potential avenues for future collaborations between researchers and technology companies. The findings highlighted the role of the privacy paradox, as participants expressed concerns about numerous data-related activities that they have likely permitted upon agreeing to use the apps and websites from which they were enrolled. Thus, researchers and ethical boards should consider these moderate levels of trust, privacy concerns, and willingness to share data when evaluating the risks and benefits of such partnerships. Meanwhile, other perspectives, such as legal and technical insights, should also be considered. When researchers can affect decision making, apps used for research purposes should be designed to decrease the extent to which participants must agree to data collection activities that concern them. For example, allowing participants to opt in or out of different aspects and providing multimedia (ie, “gist”) rather than text-based (ie, “verbatim”) explanations of the terms might reduce the privacy paradox in online and mobile research. For any secondary collection of data from apps, researchers should provide potential participants control over the types of data shared to the greatest extent possible, given the varying levels of concerns across different types of data that apps might have access to. Further research in this area is critical, particularly in the light of ongoing public awareness of and debate about technology and privacy [44,45].