## Advancing Research Transparency: Exploring the Relationship Between Diversity and Innovation### AbstractA split-sample/dual method research protocol is demonstrated to increase transparency while reducing the likelihood of false discovery. We apply the protocol to examine whether diversity in ownership teams increases or decreases the likelihood of a firm reporting a novel innovation, using data from the 2018 United States Census Bureau's Annual Business Survey. Transparency is enhanced in three ways:- All specification testing and identifying potentially productive models are conducted in an exploratory subsample.- The validity of hypothesis test statistics from "de novo" estimation is preserved due to this separation. - All findings are publicly documented in a Registered Report and this journal publication.Bayesian estimation procedures that leverage information from the exploratory stage and incorporate it into confirmatory stage estimation replace traditional frequentist null hypothesis significance testing. In addition to increasing statistical power by using information from the full sample, Bayesian methods directly estimate a probability distribution for the magnitude of an effect, allowing much richer inference. Estimated magnitudes of diversity along academic discipline, race, ethnicity, and foreign-born status dimensions are positively associated with innovation. A maximally diverse ownership team on these dimensions would be roughly six times more likely than a homophilic team to report new-to-market innovation.### IntroductionCredibility concerns extend beyond scientific research, undermining the quality of public debate on important issues such as climate change, public health, and the value or triviality of promoting diversity. The admonition to "trust science" loses weight when research findings are not reproducible, fail to replicate in related studies, and aren't fully transparent in their derivation or testing of alternative hypotheses.This article presents a proof of concept for a split-sample/dual method research design applied to the controversial topic: the connection between the diversity of ownership teams and business innovation. Our protocol addresses two significant contributors to the fragility of findings:- By splitting the dataset into exploratory and confirmatory samples, the validity of statistics assuming "de novo" tests are preserved.- The validity of comparing numerous alternatives is preserved through false discovery rate and family-wise error rate corrections.Our protocol replaces frequentist methods with Bayesian methods in the confirmatory stage of the analysis. This modification leverages information learned during the exploratory stage and uses these estimates as weakly informative priors in the confirmatory phase.The protocol also provides an implementable solution to the problem of data-dependent analysis that plagues research using observational or secondary data. This problem, where unreported repeated specification testing produces fragile, often unreplicable findings, has long been recognized. Pre-registration has since been widely adopted for randomized controlled trials, but "it is unclear how to apply pre-registration to the analyses of existing data, which account for the vast majority of social science. Development of practices appropriate for existing data... is a priority."### Diversity Findings Vulnerable to False DiscoveryThe empirical problem we address assesses how diversity within ownership teams is associated with higher or lower probability of reporting new-to-market innovation. The data is from the 2018 Annual Business Survey (ABS) produced jointly by the United States (U.S.) Census Bureau and the National Center for Science and Engineering Statistics. ABS replaced the Survey of Business Owners for employer firms, added the innovation module from the former Business R&D and Innovation Survey, and an R&D module for microbusinesses with fewer than ten employees. Demographic and background information for up to four owners per firm is collected.The hypothesis that diversity in ownership teams increases the incidence of innovation derives from a combinatorial conception of innovation: the bringing together of seemingly unrelated ideas. For example, business owners from different academic specializations can be expected to combine ideas from different domains.Findings at the firm level of whether diversity or homophily is more likely to promote innovation are mixed. The hypothesis that diversity hinders innovation derives from the role that homophily may play in facilitating the flow of information. Cognitive conflicts, where a divergence of ideas arises due to different experiences, may resolve as performance-enhancing synthesis.### Data and Analytical Details- Ownership fractionalization (OF) refers to the diversity of ownership shares within a team, incorporating the number of owners and the size of their shares.- A composite ownership fractionalization (COF) index is used to measure diversity across multiple dimensions simultaneously.- The sample was split 35%/65% between exploratory and confirmatory samples to maximize power in the confirmatory sample. - The exploratory analysis included 127 different diversity measures, subjected to false discovery rate and family-wise error rate corrections.- Bayesian estimation was used in the confirmatory stage, incorporating weakly informative priors from the exploratory analysis to improve precision and inferential richness.### Results- All but one diversity measure estimated in the confirmatory analysis were composite, multidimensional measures.- Decomposition analysis revealed that Education Specialization was the diversity dimension with the strongest association with innovation, followed by Race, Foreign-born Status, and Ethnicity.- A maximally diverse ownership team on these dimensions would be roughly six times more likely than a homophilic team to report new-to-market innovation.### Discussion- Findings cannot be summarily discounted as cherry-picked because all specification testing results were published in a Registered Report providing a pre-analysis plan. Through separation of data for exploratory and confirmatory analyses, transparency was increased.- Limitations of the study include that causality has not been established, and future research should explore designs to control for endogeneity and extend analysis beyond self-reported innovation measures.### Policy ImplicationsThe strong association between diversity in terms of race, ethnicity, foreign-born status, and education specialization on innovation has policy implications regarding immigration and affirmative action policy, but discussion should await establishment of causation.### ConclusionThe split-sample/dual method approach increases research transparency, reduces the likelihood of false discovery, and provides richer inference. Expanding its use could enhance the quality of public debate on contentious issues and the overall credibility of applied research.