Analyzing Customer Satisfaction of Apple AirPods Using Exploratory Data Analysis and Classification Techniques (Part 2)
In Part 1, I went through the statistics regarding the industry, Apple, and AirPods. In this article, by using Python, I will focus on a more technical analysis of my survey data to help us understand how customers are satisfied with their AirPods.
Objectives
The survey was conducted to identify the satisfaction level towards Apple AirPods among college students in Durham, NC. In my assessment, I also identified AirPods users' demographics within the Durham area, the frequency of using the devices, activities they perform while using devices, and the features that these respondents consider important during their use of the device. By analyzing the respondents’ answers, we’d be able to provide insights on the correlations of the important features and the current satisfactory level. We aim to provide recommendations for the manufacturers to improve some of the features resulting from the analysis.
Survey implementation
- Methodology: Email survey to a selected group (college students and young residents in Durham, NC). The survey link comprised of questionnaires was sent to the group by email.
- Process:
- Questions design
- Sample selection and justification
- Implementation
- Data Collection and Analysis
- Review and Control
- Survey structure:
- Part 1: Diagnostic questions
- Part 2: Satisfaction questions
- Part 3: Demographics
Survey data understanding
The survey output was converted into a dataset that can be used for further analysis. A template of this survey can be found here. The resulted dataset includes 89 observations (for each survey response) and 41 columns (for each survey question).
Results analysis
Demographics
During the survey period, I was able to collect data from 89 respondents. The survey data implies that young people are the dominant age group using AirPods: 60% of people from 18–25 and 34% from 26 to 35. This is reasonable if we look further at the demographic analysis, which shows the dominance of full-time college students at 75%. Other findings worth mentioned are: males are slightly more involved in the survey than females; graduate students double undergraduate students in quantity; and 100% of the selected group has a business major.
Behaviors analysis
Although AirPods can work with non-Apple products, most AirPods users are iPhone owners — 95%. Even when Apple launched its newest third-generation, called AirPods Pro, more than a year ago, the survey data suggests the extensive coverage of the first generation among their loyal customers, 43% of respondents, followed by the second generation 38%. AirPods Pro, meanwhile, is used by only 19% of respondents. Another explanation is that over 75% of AirPods users purchased their products years ago when the first generation was introduced.
Second, data shows that people start wearing this device in the morning, starting a working day or going to school, and come back using it again in the late afternoon when they spend time exercising.
Diving deeper at below specific activities aligned with the AirPods, we know that working/studying, exercising, and walking are among the top three respondents. This verifies our above assumption.
Last, as most people use AirPods 2–3 times per day, they usually spend 1–4 hours with this device.
Important features analysis
The top 3 features, Battery life, Secure fit in ears, and Sound quality, receive the most attention from survey participants. Meanwhile, people see the two new features introduced by AirPods Pro: Noise cancellation and Water resistance, as less critical.
Satisfaction analysis
In general, Apple AirPods products received a very high level of satisfaction from consumers. 97.7% of surveyed customers said they were completely satisfied, very satisfied, or satisfied with the products. Regarding the overall satisfaction for each AirPods version, the outcomes have no much difference.
Diving deep into customers’ satisfaction level for individual feature/function, we witness two patterns. First, people are highly satisfied with: Design, Charging speed, Bluetooth pairing, and Battery life. Second, people are not much satisfied with: Security Fit in Ears, Tapping function, Water resistance, and Noise cancellation. Data in the table show the mean satisfaction score for each feature input by customers (1–5 in the order of decreasing satisfaction).
Correlation analysis
To see which variables are likely to affect the customer’s overall satisfaction, I ran a correlation analysis of our independent variables against our dependent variable, overall satisfaction. This analysis ended up with a list of variables of interest that had the highest correlations with overall satisfaction.
In order of highest correlation in absolute value with overall satisfaction, these variables are:
For these independent variables, the next step to further analyze the associated relationships with our dependent variable was to create scatter plots. I picked four features for this visualization: Design, Sound quality, Secure fit in ears, and Battery life. A typical pattern can be observed. The higher satisfied customers are about each of these four, the higher overall satisfied they are about the AirPods overall.
To learn more about the relationships within independent variables and overall satisfaction, I built different three-dimensional plots. I took into account such demographic attributes as Age, Gender, and Employment status. I also considered the smartphone operating system people are using and how long they usually use their AirPods. When inspecting the two variables, Gender, and Satisfaction — Sound quality with overall satisfaction, we can see that male users are less satisfied with sound quality but still highly satisfied with the overall product. Meanwhile, female customers are more sensitive with sound quality in scoring the overall satisfaction. Another pattern resulting from an interaction analysis using Satisfaction — Design and Age in relationships with overall satisfaction suggests that older customers, 26–35, are less satisfied with Design and, therefore, the overall product.
Modeling and Importance of features
Based on the EDA and correlation analysis, I considered two potential models using machine learning techniques: Decision tree and Random forest. For the performance evaluation part, I then compared these two models by their accuracy.
Model 1: Decision tree
Decision tree is a popular model used in the classification of discrete target variables using machine learning. A decision tree is a binary tree flowchart where each node splits a group of observations according to some feature variable. A decision tree aims to split your data into subgroups such that every element in each group belongs to the same category.
In this case, each node's “value” row tells us how many-sorted observations into that node fall into each of our four categories: Completely satisfied, Very satisfied, Satisfied, and Somewhat satisfied. We can see that Satisfaction_Design and Importance_Sound_quality are two of the most important features that help identify and classify categories. The picture below shows the first four nods. A full decision tree can be found here.
We have performance metrics as below:
Model 2: Random forest
Random forests are an ensemble learning technique for classification, regression, and other tasks that operate by constructing a multitude of decision trees at training time and outputting the class that is the mode of the classes (classification) or mean/average prediction (regression) of the individual trees.
The model then chooses the mode of all of the predictions of each decision tree. Relying on a “majority wins” model reduces the risk of error from an individual tree. Using Random forest in this case, we have performance metrics as below:
By comparing the two models, the random forest seems to yield a higher accuracy level.
Importance of features: Below, I graphed the feature importance based on the Random Forest model. The top 4 features are Satisfaction — Design, Satisfaction — Secure fit in-ears, Satisfaction — Battery life, and Satisfaction — Sound quality.
Additional analysis — Voice of customers
If you could change one thing about the product, what would it be? From the survey, customers want to improve Battery life, Sound quality, Bluetooth paring process, and Tapping function.
Deployment
The respondents’ overall satisfaction level towards the AirPods is relatively high (97.7% of customers were completely satisfied, very satisfied, or satisfied with the products). This is the same as the previously published survey, reinforcing that AirPods can bring a high satisfaction level to the users, explaining the device’s dominant market share.
From the modeling analysis using machine learning techniques, we can see that the satisfactory level of Design will impact the overall satisfaction most. Apple can improve the overall satisfaction of the device by improving customers’ satisfaction with this design feature.
The most important feature of the device is Battery life (mean = 1.39). However, this feature’s satisfaction level is moderate (mean = 2.05), indicating an area for improvement. Satisfaction over battery life also impacts the overall satisfaction of the device. We recommend Apple to improve this feature for a higher satisfaction level. Some recommendations on improving battery include increasing battery life, showing battery life on AirPods case rather than having to paired to phone to see battery life, or having a portable charging case.
The other features: Secure fit in ears and Sound quality also tends to impact the customers’ overall satisfaction on AirPods, especially when they use the device most for Working/Studying and Doing exercise. In 2019, Apple listened to the customer’s voice by launching the new Apple AirPods Pro version with two new features: noise cancellation and improved fit in-ears. However, the survey suggests that people are still concerned about them. For that reason, Apple should invest more in improving these important functions to enhance the customers’ satisfaction level.
Although Tapping function is not considered a significant feature, it receives the respondents’ lowest satisfaction level. Respondents also provide insights on how to improve Tapping function, such as changing the volume with it, or more programs installed, or tapping on one ear can also work on the other ear. The company can invest more to improve this Tapping function since that would affect customer satisfaction over this feature, improving the overall satisfaction level.
However, this analysis has some limitations. The sample size of our survey is limited (89 respondents). Hence the sample may not be a good representative of the population. Besides, a small number of respondents use the latest version — AirPods Pro (19%), which may impact the overall conclusion regarding the recommendations for the future product.
You can also take a look at my brief visualization on Public Tableau.
