Unlimited Data Maturity Assessments with ChatGPT
Data maturity is crucial, but expensive assessments can feel out of reach. What if there was a way to generate them instantly with AI?
For many years we have made a free Data Maturity Assessment available on our website. The reason for this is simple – many Data Teams want a quick baseline of their maturity and don’t want to spend tens of thousands of dollars to buy one from a consultant.
Our Data Maturity Assessment was hosted on Typeform (an online survey tool), and then stitched together on the back end with Google Sheets and Zapier to automatically push a result based on user input.
We used this to create our Data Maturity research study, as well as to provide consulting sessions and tailored training to our customers.
Data Maturity Assessments in the age of AI
Since ChatGPT was launched it has become ever easier to replace the work you’d pay a Data Maturity consultant to do. As far as I’m concerned, Data Maturity Assessments are a prime example of a basic measurement tool that can be automated. Read on to learn how to make unlimited Data Maturity Assessments with ChatGPT.
ChatGPT has been trained on more than enough information to create a maturity assessment. You can even choose the flavour – whether your firm follows DAMA, DCAM, or any of the other models – you can get a pretty good assessment in seconds.
As a result, it is time to retire the Cognopia Data Maturity Assessment (RIP, old friend) and respect the fact that we can do better in 2024 with less effort (and I can stop paying Typeform/Zapier and Google to create reports that no-one reads).
Benefits of creating Data Maturity Assessments with ChatGPT
I see the following benefits of using ChatGPT to produce and administer your Data Maturity Assessments:
- It is FAST – you can generate an enormous amount of questions in minutes
- It is FLEXIBLE – a few tweaks to the prompt and you’ll get vastly different output, fit for your business needs
- You can chat with GPT to get additional explanations on the context BEHIND the question – e.g. if you do not know what a Data Steward is, ask ChatGPT during the survey
- It is FILTHY CHEAP – you don’t need to hire a consultant or spend thousands on a DCAM membership – $20 a month on a ChatGPT subscription gets you unlimited questions
This post will walk you through how the prompt is constructed and why this method works, or you can skip straight to the copy/paste prompt by clicking the button below:
Data Maturity Prompt Engineering 101
Prompt engineering is a fancy way of saying “Writing a prompt that has repeatable outputs from a GPT”.
I spent a morning over the weekend tweaking this prompt – it is by no means perfect, and I would recommend playing around with it to learn what the various components do AND to improve the output.
The best thing about using GPT to produce these maturity assessments is the speed and volume of output you can create. Take a 30 minute coffee break and you can produce something remarkably tailored to your needs..
Data Maturity Assessment AI Prompt setup:
The setup phase tells the GPT what role to play, and it introduces the concept of variables. Variables allow us to re-use the same generic prompt whilst tweaking the output by changing only the variable.
For our needs, I’ve selected 4 variables to use:
- {{company_size}}: Size of the company (e.g., small, medium, large).
- {{industry}}: Industry sector (e.g., healthcare, finance, manufacturing).
- {{data management framework}}: e.g. DAMA, DCAM, CMMI (default to DAMA if not specified)
- {{n}}: Number of survey questions (default to 25 if not specified).
You can go wild with these if you want. I’m not sure the industry or company size variables are changing much in the output so far. I chose to keep these in so you can see the general idea and change them up.
If a variable is NOT helping you get a better output, drop it or change it.
Telling your Data Maturity AI what role to play:
We need to give the context so we get the AI to produce the outcome we need. This context includes our variables, the role we need our AI to play, and the general parameters of what we need it to do:
You are an expert Data Management Consultant for a {{company_size}} organisation in the {{industry}} industry, and you are assisting in the creation of a Data Maturity Survey to measure their current Data Maturity. Generate a set of {{n}} survey questions, based on {{data management framework}} to send to be used as a quiz to measure their current Data Maturity against this framework. Prioritise critical data management knowledge areas in the chosen framework, ensure the questions cover a broad range of these knowledge areas, and tailor the questions to focus on any industry-specific concerns.
Here we are instructing the AI to play the role of a Data Management Consultant, and specifying what task this consultant is performing. We have used the variables to set the scope for the engagement (company size, industry, framework and number of questions to produce).
Data Maturity Questions - Generated by AI
Now our AI knows what role to play, it needs to be given explicit instructions about how to create our questions. We do this using the following instructions:
The questions should be in simple, business-friendly language, addressing any known organisational contexts, and suitable for non-technical respondents.
This is a relatively simple setup line – trying to set the tone of the question, and align them with the industry/company size. We need this to be answerable by a non-technical audience.
Each question should focus on a specific core competency within the knowledge area being tested. The greater {{n}} is, the broader your questions on each knowledge area can be and the more questions on the subject you can ask. The collection of question should be MECE. Because the number I ask for could be different each time, take this into account as you ensure that you have complete coverage of concepts. Therefore, some outputs may have more highly themed questions (fewer questions) and some may be more granular (more questions).
Now we take it a level deeper. Each of the Data Management Maturity Frameworks has their own set of important knowledge areas. ChatGPT is well aware of these (based on my testing). I don’t know how many questions each user will need to generate, so I set some instructions to make them generic and broad (if the user selects a small number of questions) vs atomic and detailed (if we have more questions to play with).
The questions will be rated by survey respondents on a scale of 1 through 5, where 1 will be the worst case and 5 being the best case. The following is an example:
Q: How is Data Quality viewed by your organisation?
1: It is rarely or never thought of
2: Data is viewed as poor quality, but no processes are in place to fix it
3: Data Quality is managed by the IT team, centrally
4: Data Quality issues are handled by a dedicated team that rectify and improve our data
5: Data Quality is seen as a responsibility across all roles in the organisation and everyone plays their part to maintain or improve the data they create and use
For each question, you will need to create a sensible scale representing each level of data maturity in the chosen framework. The scale will be used to calculate data maturity at the knowledge area and overall level at a later step in the process. Each situation could need different worst case and best case language. You will need to think that through carefully.
Now we get into the specifics. We will generate questions that follow the broad CMMI maturity assessment score (1-5). Each answer needs to represent a step-up in maturity.
We provide an example of the format for the questions and their answer text (this is one of our existing Data Maturity Assessment questions). Giving an example helps the AI create a consistent output.
Telling the AI to think things through carefully usually results in a better output.
Data Maturity Questions - capturing our variables:
Now we need to wrap the prompt up with space for the user to edit the prompt variables, and a final set of recommendations on our formatting expectations:
If some variables are unknown or not provided, proceed using general assumptions relevant to similar organizations.
Always output in markdown
Variables:
* {{company_size}}: Size of the company (e.g., small, medium, large).
* {{industry}}: Industry sector (e.g., healthcare, finance, manufacturing).
* {{data management framework}}: e.g. DAMA, DCAM, CMMI (default to DAMA if not specified)
* {{n}}: Number of survey questions (default to 25 if not specified).
DO NOT output any text before or after the question list
The final instruction tries to lose the “chatty” nature of GPT.
Data Maturity Assessment - the full AI prompt
You probably didn’t want a lecture on AI Prompt Engineering – instead you wanted the full prompt so you can cut/paste it into your preferred GPT and get going. Here it is:
You are an expert Data Management Consultant for a {{company_size}} organisation in the {{industry}} industry, and you are assisting in the creation of a Data Maturity Survey to measure their current Data Maturity. Generate a set of {{n}} survey questions, based on {{data management framework}} to send to be used as a quiz to measure their current Data Maturity against this framework. Prioritise critical data management knowledge areas in the chosen framework, ensure the questions cover a broad range of these knowledge areas, and tailor the questions to focus on any industry-specific concerns.
The questions should be in simple, business-friendly language, addressing any known organisational contexts, and suitable for non-technical respondents.
Each question should focus on a specific core competency within the knowledge area being tested. The greater {{n}} is, the broader your questions on each knowledge area can be and the more questions on the subject you can ask. The collection of question should be MECE. Because the number I ask for could be different each time, take this into account as you ensure that you have complete coverage of concepts. Therefore, some outputs may have more highly themed questions (fewer questions) and some may be more granular (more questions).
The questions will be rated by survey respondents on a scale of 1 through 5, where 1 will be the worst case and 5 being the best case. The following is an example:
Q: How is Data Quality viewed by your organisation?
1: It is rarely or never thought of
2: Data is viewed as poor quality, but no processes are in place to fix it
3: Data Quality is managed by the IT team, centrally
4: Data Quality issues are handled by a dedicated team that rectify and improve our data
5: Data Quality is seen as a responsibility across all roles in the organisation and everyone plays their part to maintain or improve the data they create and use
For each question, you will need to create a sensible scale representing each level of data maturity in the chosen framework. The scale will be used to calculate data maturity at the knowledge area and overall level at a later step in the process. Each situation could need different worst case and best case language. You will need to think that through carefully.
If some variables are unknown or not provided, proceed using general assumptions relevant to similar organizations.
Always output in markdown
Variables:
* {{company_size}}: Size of the company (e.g., small, medium, large).
* {{industry}}: Industry sector (e.g., healthcare, finance, manufacturing).
* {{data management framework}}: e.g. DAMA, DCAM, CMMI (default to DAMA if not specified)
* {{n}}: Number of survey questions (default to 25 if not specified).
DO NOT output any text before or after the question list
Disclaimer: use this with caution. I’ve thrown it together quickly and tested it on ChatGPT (currently version 4o). Like any other output you get from an AI chatbot, you need to use your brain and ask “Is this a reasonable question?” and “Is this important to our business?” before using it.
Improving the Data Management Maturity prompt
I’m unlikely to put any more effort into this prompt. I created it so we can switch off our free Data Maturity Assessment, and because I think this is the future. Produce as many Data Maturity Surveys as you wish, for free. The future is unlimited.
If I did want to improve this further, here are some areas I think are worth considering:
Improving Data Maturity Prompt Variables
The Variables are the first place I would play with. I suspect the Industry and Company size inputs are adding limited value right now.
Knowledge Areas: You could experiment here by setting the specific Knowledge Areas you wish to assess (e.g. “Data Governance, Data Architecture” etc) so the questions being output are more focused on your needs.
Known Problems: You could use a variable to include any known issues you think exist. CAUTION if you are a novice then this could do more harm than good.
Industry-specific regulations: You could easily generate a list of questions targeting your specific industry/geography regulations by adding a variable and asking to capture this in the output.
Improving Data Maturity Prompt Questions
This would be harder, but you might feel the wording or language in our example question and answer choice is weak. If you have a format or style, provide that as an example instead.
Automating Data Maturity Assessments Entirely
What if you were really lazy and couldn’t be bothered to even copy and paste a Data Maturity Prompt into your chosen GPT? What if someone had created a custom GPT to just prompt you for the variables and then walk you through the quiz?
Well, you’re in luck – I also put together a Custom GPT that you can use to simply quiz yourself by answering questions. Note: this has not gone through extensive testing – if you find issues with it feel free to let me know, but I can’t promise to spend a long time fixing them as you’ve got the full prompt used above.
This Custom GPT will create as many Data Maturity Assessments as you want, provide a score and recommendations, and could help you boost your Data Maturity (or at least shine a light on your current issues).
