The general public reads the dramatic implications printed in newspapers, published on the web, or featured in the lifestyle montage on the TV news.Bandura cognitive learning theory the public hears the lofty statements about how this one study changes the meaning of human life or predicts the fate of the universe and concludes, “that just doesn’t make sense that you get there from here.” they may not understand probability theory, but they understand bunkum.Bandura cognitive learning theory consequently, over time, p<.05 has taken on a different meaning for them.

Why does the popular press seem to publish without hesitation these kinds of pop-science stories? Print publishers that were once staffed with an army of trained journalists now operate with a skeleton crew, having lost most of the advertising revenue to online outlets.Bandura cognitive learning theory too often, those publishers will now gladly reprint a press release from a reputable university as if it were news. Online news aggregators are hungry for any content, but need a headline that gets clicks and, if the study is not interesting enough by itself, will write a more provocative title.Bandura cognitive learning theory and TV news just follows the clickstream of trending links.

Why do people read these stories? Because they are interesting. People like to read interesting stories, whether or not they are true.Bandura cognitive learning theory personally speaking, I miss bigfoot. Was bigfoot the work of a huckster with a set of modified snow shoes and a nikon camera with vaseline on the lens?Bandura cognitive learning theory maybe. But bigfoot never gave anyone a false diagnosis of cancer or misguided greece from a tough economy to an economic disaster. Bigfoot would never try to make a 100-year-forward prediction based on only 100 years of data.Bandura cognitive learning theory bigfoot would never do such things.

How did these kinds of studies ever get published in academic journals? Because publish-or-perish tenure track research jobs require publishing anything that can be published?Bandura cognitive learning theory if you have to run the study with 11 slightly different variations or build the statistical model using something akin to a monte carlo assortment of covariates to get to p<.05, that’s what you have to do?Bandura cognitive learning theory

I tend to disagree that p-hacking or tenure-tracking are the main causes of the kinds of research that have given a bad name to p<.05. In the physical sciences, there is straight-up fraud and, in the social sciences, jiggle-the-handle discoveries that will never be replicated by anyone anywhere.Bandura cognitive learning theory open data (raw data with all code transformations) and open methods (all of your statistical modeling including anything you tested) could fix those problems immediately.Bandura cognitive learning theory fraud and honest mistakes could be zeroed out quickly.

I am reminded of when I was a regular reviewer for an academic journal twenty years ago.Bandura cognitive learning theory I received paper copies of papers for review. There was a method to manually reviewing statistical tables to verify degrees of freedom were calculated correctly, critical values precise, and all of that.Bandura cognitive learning theory in essence, you were reverse engineering the statistical results starting with only the experimental design parameters and the final output.Bandura cognitive learning theory I cannot imagine doing that now. A complete end-to-end review of all code and statistical modeling starting with the raw data is standard operating procedure in the private sector world of analytics.Bandura cognitive learning theory this should be standard practice for all academic peer reviews…if you can find the people to do it.

Another factor could be that the tenure system, itself, is collapsing.Bandura cognitive learning theory universities are replacing tenure track positions with temporary teaching or adjunct faculty positions, a phenomenon called the “ adjunct crisis.” thus, the pool of people eager to pursue research or serve in the kinds of roles peer-reviewed science requires is drying up.Bandura cognitive learning theory it is possible that the number and quality of reviewers to support the peer review process has declined and consequently more papers of questionable merit can slip through the process.Bandura cognitive learning theory and that doesn’t even begin to account for the so-called “open” internet journals that have a back-end, pay-to-publish business model.

Another reason could be the shift away from pure empiricism to theory-making and social impact statements.Bandura cognitive learning theory this has surely driven much of the problem in the social sciences that are now in the midst of what is called the "replication crisis." well-regarded academic journals that once would only publish studies in which the authors themselves had replicated findings multiple times with different samples now seem more interested in the discussion than the results.Bandura cognitive learning theory to meet that demand, researchers now develop broad theories around small scale sets of facts and then expound upon those theories with real-world implications and predictions of social impact.Bandura cognitive learning theory

That trend, itself, may be due to the fact that universities have pushed researchers to identify ways to commercialize their research, encouraging entrepreneurial opportunities to monetize basic science.Bandura cognitive learning theory for studies without commercial possibilities, researchers are encouraged to emphasize the social impact and ways in which each analysis will make the world a better place.Bandura cognitive learning theory that is certainly a part of the problem.

Over the past few decades, researchers have tried to improve our understanding of p<.05 with demands for effect size calculations that reveal the weakness in a statistically significant finding and meta-analytic reviews that attempt to summarize multiple statistically-significant (and non-significant) findings, forcing a close look at the generalizability of individual studies.Bandura cognitive learning theory the new push for replication studies has done the same in a more direct manner. But none of those efforts will alleviate the broader systematic challenges facing the support system for basic science research.Bandura cognitive learning theory

Yet another factor affecting how we regard p<.05 is the availability of big, real-world open data sets. My undergraduate and graduate school research was mostly confined to carefully designed experimental studies and relatively small correlational studies.Bandura cognitive learning theory an n of 30 human beings was standard in experiments. A thousand subjects in a longitudinal study were together “big” data. In that world, p<.05 had a well-defined meaning.Bandura cognitive learning theory

When I first had access to mainframe data with 30,000 respondents for a survey, I could not help but start with a correlational analysis of all of the variables.Bandura cognitive learning theory but I was confused by the output. I saw cell after cell of .01 and .01 and could not figure out where the correlation was in the output. I eventually realized, to my surprise, that the correlation was .01 and the p value was .01 and my understanding of statistics was never the same.Bandura cognitive learning theory

One of the functions of probability theory is to estimate the degree to which a sample is representative of a population. What is the likelihood that this sample could have been obtained from this population?Bandura cognitive learning theory these days, in so many domains, we have ready access to exceptionally large real-world data sets. We know the “sample” could have come from that larger “population” because we extracted the sample ourselves.Bandura cognitive learning theory there is less need to rely on p<.05 to support a generalization because we have the “population” estimates. Of course, big data are not all of the data and even exceptionally large data sets can be biased, but the question as to whether the sample is like a “population” is easier to answer when you have the “population” to test it.Bandura cognitive learning theory

These problems are compounded by high-speed automated analytics where tens of thousands of regression models are calculated to re-price every sku in a department store chain overnight or machine learning models that generate countless hidden nodes in a neural network model.Bandura cognitive learning theory in the content of production analytics, p<.05 has virtually no meaning and there is no simple bonferroni adjustment for that.

Elliot inman, ph.D., is a manager of software development for SAS® global hosting and US professional services.Bandura cognitive learning theory over the past 25 years, he has analyzed a wide variety of data in areas as diverse as the effectiveness of print and digital advertising, social service outcomes analysis, healthcare claims analysis, employee safety, educational achievement, clinical trial monitoring, sales forecasting, risk-scoring and fraud analytics, general survey analysis, performance benchmarking, product pricing, text processing, and basic scientific research on human memory and cognitive processes.Bandura cognitive learning theory after completing his undergraduate degree at north carolina state university, he went on to earn a ph.D. In experimental psychology from the university of kentucky in 1997.Bandura cognitive learning theory in 2005, he started at SAS as an analytical consultant. In 2010, he joined SAS solutions ondemand (now SAS global hosting), SAS’ high performance cloud computing center.Bandura cognitive learning theory his current focus is on implementing SAS visual analytics to provide non-statisticians deeper insight into the results of data mining and machine learning models.Bandura cognitive learning theory he leads the visual analytics data visualization team for SAS' gatheriq data for good platform. In addition to his work at SAS, he has led makerspace and creative coding workshops for college students who want to explore a different way of using technology to do cool and useful things.Bandura cognitive learning theory