As some readers may recall, my colleague CoreySchimpf and I published an article in 2022 identifying a new area of methods development, called smart and approachable methods or AM-Smart for short. CLICK HERE for an open access copy of the article at IJSRM.
What are AM-Smart methods?
AM-Smart platforms do so through nine design features. They are (1) bespoke tools that (2) involve a single or small network of interrelated (mostly computational) methods. They also (3) embed distributed expertise, (4) scaffold methods use, (5) provide rapid and formative feedback, (6) leverage visual reasoning, (7) enable productive failure, and (8) promote user-driven inquiry; all while (9) counting as rigorous and reliable tools.
Examples include R-shiny programs, computational modeling and statistical apps, public-sector data management platforms, data visualization tools, and smart phone apps.
What about ChatGPT for advanced data analysis?
What we had not thought about was that advances such as ChatGPT also constituted a form of AM-Smart methods. That is until today, when I saw an advertisement for a 2-day instats workshop by Peter Gruber, titled, appropriately enough, UsingChatGPT for Advanced Data Analysis. Here is the brief description of the workshop:
This 2-day workshop teaches researchers, from PhD students to professors, how to use ChatGPT and its Advanced Data Analytics tool for statistical analysis without writing a line of code or even knowing how to use a statistics program. The seminar covers a range of topics from data preparation and descriptive statistics to regression analysis, advanced statistical tests and visualisation. Special emphasis is placed on understanding the workings and limits of AI models such as ChatGPT and reflecting on its implications for data analysis.
In addition to the description of the workshop, what caught my eye was the instats description of Gruber’s work around teaching methods.
Dr Gruber has been an early adopter of Large Language Models for teaching and research and leads the efforts of USI to integrate LLMs into the curriculum. He has been teaching statistics, financial econometrics and numerical methods with MATLAB, Python and R since 2005 at USI, University of St. Gallen, University of Geneva and Bocconi.
This was insightful, because it suggested the approach that is very much driving the AM-Smart movement. Which is super cool.
This is not the only workshop!!!
To lean more, I started exploring on Google if other such workshops or courses were being offered. They are being offered! Lots of them. A whole new field of study and pedagogy seems to have emerged over night! Digital sociology and data science have a lot to think about!
Which led me to next ask, What is ChatGPT for advanced data analysis?
What is ChatGPT for advanced data analysis?
Let’s not get too excited!!!!
As Corey and I outlined in our articleon AM-Smart methods, while offering new possibilities for data analysis, critical reflection on AM-Smart platforms reveals considerable unevenness in these design features, which hamper their effectiveness.
A rigorous research agenda is vital.
A case in point is ChatGPT Advanced Data Analysis.
· First, while currently free, it is not clear if it will not stay that way for long. This is potentially a major issue and goes against the open-access environment of current science and programming platforms such as R.
· Second, it is not a bespoke suite of methods. It is unclear what sorts of algorithms are used in the background to do the work. What are the limits of these methods?
· Third, it is not as user-friendly as it suggests. Part of the whole point of AM-Smart methods is ease of access. For example, there are limits to what data can be used and how it can be used. I am sure these issues will be addressed going forward, but presently the plug-in seems to be very much at the ground level.
· There is still a significant learning curve to using it, including getting the plug-in to work.
· Then there is the issue of data safety and access globally!
Then there is the question
of pedagogy and value for teaching. It is key to have methods experts such as Peter Gruber involved in the development of AM-Smart methods, such as ChatGPT. There are so many questions to explore. Is this a good way to learn statistics? Is fast statistics good statistics? What about gaming? Is that a good way to use methods? For example, I am
increasingly worried about students not learning the math behind the methods
they use, in the case of numerical analysis, and, in turn, the craft of qualitative
inquiry, as in the case of ethnography.
Conclusion?
My short conclusion is that ChatGPT is a good example of AM-Smart methods but comes with lots of concerns that require the methods community to engage with it before endorsing it.
I do not make this point nonchalantly, as these sorts of smart tools are not the necessary future of methods. There is no inevitability here. Instead, there is lots of room for creative but critical interrogation and development, both for educational purposes and for the wider usage of methods in research, policy and practice.
