Data Visualization and Storytelling
Open in the Padhaiko app5.1 Principles of effective visualization and dashboard design
A visualization succeeds when a viewer grasps the pattern, comparison, or anomaly accurately in seconds without misreading scale or drowning in clutter.
Core Principles of Effective Visualization
- Match Chart Type to Variable Types:
- Trend over time: Line chart.
- Category comparison: Horizontal or vertical bar chart (zero baseline required!).
- Single distribution: Histogram, KDE, or box/violin plot.
- Relationship between two continuous variables: Scatter plot.
- Part-to-whole: Stacked bar or 100% stacked bar (avoid 3D pie charts or pies with more than 5 slices).
- Maximize Data-Ink Ratio (Edward Tufte): Remove redundant 3D effects, heavy dark gridlines, chartjunk, and decorative borders so visual ink encodes data values.
- Honest Scales and Integrity:
- Bar charts must start at zero (
0) because bar length encodes magnitude; truncating a bar axis exaggerates small differences. - Line charts may zoom to a narrower y-range when small fluctuations are meaningful, provided the axis range is clearly labeled.
- Bar charts must start at zero (
- Color and Accessibility: Use sequential palettes (
viridis,Blues) for ordered magnitudes, diverging palettes (coolwarm,RdBu) for values centered on zero/target, and qualitative palettes (tab10) for distinct categories—preferring colorblind-safe palettes and direct labels.
Dashboard Design Principles
Effective Dashboard Architecture
- Inverted-Pyramid Hierarchy: Place high-level KPI cards (key totals, YoY deltas) at the top, trend and breakdown charts in the middle, and granular filterable tables at the bottom.
- 5-Second Rule & Focus: Limit a single dashboard view to 4–7 cohesive visuals answering one audience's decisions rather than cramming every metric onto one screen.
- Consistent Formatting & Context: Align time axes vertically, keep unit labels/colors consistent across panels, and always show comparison baselines (target, prior period, or average).
5.2 Visualization with Matplotlib: Line, bar, histogram, scatter, subplots
Matplotlib is Python's foundational plotting library. Using its object-oriented API (fig, ax = plt.subplots()) instead of stateful global plt.plot() calls gives fine-grained control over multi-panel figures, styling, and automated script exports.
| Plot Type | OOP Method | Data Purpose & Key Arguments |
|---|---|---|
| Line Plot | ax.plot(x, y, marker="o", linestyle="-") | Continuous trends or time-series trajectories |
| Bar Chart | ax.bar(cats, vals) / ax.barh(...) | Comparing discrete category magnitudes (barh fits long labels cleanly) |
| Histogram | ax.hist(x, bins=20, edgecolor="black") | Frequency distribution of one continuous variable (density=True normalizes area to 1) |
| Scatter Plot | ax.scatter(x, y, c=z, s=sizes, alpha=0.7) | Joint relationship, clusters, and outliers between two numeric features |
| Subplots Grid | fig, axes = plt.subplots(2, 2, figsize=(10, 6)) | Multi-panel layouts; call fig.tight_layout() or constrained_layout=True to avoid label overlap |
import matplotlib.pyplot as plt
months = ["Jan", "Feb", "Mar", "Apr"]
revenue = [120, 145, 138, 172]
orders = [40, 48, 45, 58]
fig, axes = plt.subplots(1, 2, figsize=(10, 4), constrained_layout=True)
axes[0].plot(months, revenue, marker="o", linewidth=2)
axes[0].set_title("Monthly Revenue")
axes[0].set_ylabel("Revenue (k NPR)")
axes[1].bar(months, orders, color="steelblue")
axes[1].set_title("Monthly Orders")
axes[1].set_ylabel("Order Count")
fig.savefig("monthly_summary.png", dpi=150)5.3 Seaborn for statistical visualization: Box plot, pair plot, heat map
Seaborn builds on Matplotlib with a tidy-DataFrame-first API (data=df, x=..., y=..., hue=...) and built-in statistical aggregations (confidence intervals, kernel densities, quantiles).
Key Statistical Plots in Seaborn
- Box Plot (
sns.boxplot(data=df, x="dept", y="salary")): Displays the five-number summary—minimum non-outlier whisker,Q1(25th percentile), median (Q2),Q3(75th percentile), maximum non-outlier whisker (Q3 + 1.5*IQR), and individual outlier dots—ideal for comparing spread and skew across groups. - Pair Plot (
sns.pairplot(df, hue="species", corner=True)): Creates anN x Nmatrix of scatter plots for all numeric column pairs and histograms/KDEs along the diagonal, revealing class separability and feature correlations simultaneously. - Heat Map (
sns.heatmap(corr_matrix, annot=True, fmt=".2f", cmap="coolwarm", center=0)): Encodes a 2D matrix (such as correlation matrices or confusion matrices) as color intensity with optional exact numeric annotations.
| Aspect | Matplotlib | Seaborn |
|---|---|---|
| Abstraction Level | Low-level, imperative control over every artist, tick, spine, and annotation | High-level statistical interface working directly on Pandas DataFrames |
| Statistical Defaults | Raw arrays; manual binning, grouping, and confidence intervals | Automatic grouping (hue, col), bootstrap CIs, regression fits, correlation heatmaps |
| Interoperability | Foundation layer | Returns/accepts Matplotlib Axes (ax=axes[0]), so both mix seamlessly |
import seaborn as sns
import matplotlib.pyplot as plt
corr = df[["age", "income", "spend_score"]].corr()
fig, ax = plt.subplots(figsize=(6, 5))
sns.heatmap(corr, annot=True, fmt=".2f", cmap="coolwarm", vmin=-1, vmax=1, ax=ax)
ax.set_title("Feature Correlation Matrix")5.4 Interactive visualization using Plotly
Static PNG charts cannot zoom into dense time series, hover over outlier metadata, or filter categories live. Plotly (plotly.express and plotly.graph_objects) renders interactive HTML/JavaScript charts supporting tooltips, pan/zoom, legend toggling, and animated frames.
plotly.express vs. plotly.graph_objects
plotly.express(px): Concise one-line API analogous to Seaborn for rapid interactive exploration (px.scatter,px.line,px.bar,px.box,px.histogram), supportinghover_data=,color=,facet_col=, andanimation_frame=.plotly.graph_objects(go): Low-level composable figure builder (fig.add_trace(go.Scatter(...)),make_subplots()) for custom dual-axis dashboards, annotations, and fine-grained layout control.- Exporting & Embedding: Save standalone interactive HTML files via
fig.write_html("report.html")(zero backend server needed) or integrate directly inside Dash or Streamlit web apps.
import plotly.express as px
fig = px.scatter(
df,
x="marketing_spend",
y="revenue",
color="region",
size="customers",
hover_data=["month"],
title="Marketing Spend vs. Revenue by Region",
)
fig.update_layout(template="plotly_white")
fig.write_html("interactive_revenue.html")5.5 Visualization driven insight generation
Data storytelling transforms raw chart outputs into clear, actionable decisions for technical and non-technical stakeholders.
Three Pillars of Data Storytelling
- Data (Evidence): Verified metrics and fair visual encodings without cherry-picked ranges or distorted axes.
- Visuals (Focus): Deliberate visual hierarchy—muting background series in light gray while highlighting the critical trend in accent color, plus direct callout annotations (
ax.annotate(...)) at inflection points. - Narrative (Actionable Context): Follow the Context → Complication (Finding) → Resolution (Recommendation) arc: replace generic titles like "Sales vs. Month" with insight-driven headlines like "Q3 Mobile Sales Dropped 18% Following Checkout Redesign".
Avoiding Misleading Visualizations
- Truncated Y-Axes on Bars: Makes a 2% difference look like a 300% jump.
- Dual Y-Axes with Unrelated Scales: Can artificially cross or diverge lines; prefer two aligned subplots (
sharex=True). - Simpson's Paradox in Aggregated Charts: Always disaggregate (
hue/facet_col) when subgroup composition differs, because an aggregated trend can reverse inside every individual subgroup.
5.6 Case study: End-to-end visualization and reporting project
An end-to-end visualization project connects ingestion, cleaning, statistical summaries, multi-library charts, and automated report delivery into a repeatable script.
Raw CSV / SQL ---> Clean & Aggregate ---> Matplotlib / Seaborn (PNG) ---> HTML / PDF Report
(pandas) (groupby/KPIs) Plotly Interactive (HTML) (Jinja2 / Dash)Case Study: Retail Churn & Sales Diagnostic Report
Suppose a telecom analytics team delivers a weekly executive report from orders.csv:
- Load & Clean: Parse dates (
pd.to_datetime), filter valid transactions, compute derived KPIs (net_revenue,churn_rate,avg_order_value). - Overview Panel (Matplotlib Subplots): Export a 2x1 high-DPI summary figure comparing weekly revenue trend (with annotated campaign launch date) against category bar totals.
- Statistical Diagnostic (Seaborn): Plot a
boxplotof order size by customer tier (highlighting high-value outliers) and aheatmapof numeric feature correlations. - Interactive Drill-Down (Plotly): Generate
scatter_drilldown.htmlwithhover_data=["customer_id", "plan"]so managers inspect individual outlier accounts directly in a browser. - Insight-First Summary: Lead the final report with three quantified takeaways (e.g., "Month-to-month contracts show 3.2x higher churn than annual plans; targeted renewal offers in Week 3 recover 14% of at-risk revenue").
Worked Examples
5.1
A Seaborn boxplot summarizes response latencies (in ms) with min_raw = 5, Q1 = 20, median = 31, Q3 = 45, and max_raw = 92. Compute the box height (IQR), the theoretical upper and lower whisker fences (1.5 * IQR), and state whether 92 and 5 are drawn inside the whiskers or plotted as outlier points.
Computing Box Height (IQR) and Whisker Fences:
IQR = Q3 - Q1 = 45 - 20 = 25 ms.- Upper fence =
Q3 + 1.5 * IQR = 45 + 1.5 * 25 = 45 + 37.5 = 82.5 ms. - Lower fence =
Q1 - 1.5 * IQR = 20 - 1.5 * 25 = 20 - 37.5 = -17.5 ms.
Classifying min_raw = 5 and max_raw = 92:
- Since
5 >= -17.5,5 mslies well within the lower fence, so the lower whisker extends down to5 ms(no lower outliers). - Since
92 > 82.5,92 msexceeds the upper fence, so92 msis plotted as an outlier dot, and the upper whisker stops at the largest observed point<= 82.5 ms.
IQR = 25 ms, lower fence = -17.5 ms, upper fence = 82.5 ms. Thus 5 ms is the lower whisker endpoint (not an outlier), whereas 92 ms exceeds 82.5 ms and is plotted as an upper outlier point.
5.2
A manager creates a bar chart comparing Plan A (100 users) and Plan B (105 users) with the y-axis starting at 98 instead of 0.
- Compute the true percentage increase from Plan A to Plan B.
- Compute the visual bar-height ratio (
height_B / height_A) wheny_min = 98, and write the Matplotlib OOP fix enforcing a zero baseline with direct bar labels.
Quantifying the Visual Distortion:
- True percentage increase =
((105 - 100) / 100) * 100 = 5.0%(true ratio105 / 100 = 1.05x). - When truncated at
y_min = 98:- Plan A bar height =
100 - 98 = 2units. - Plan B bar height =
105 - 98 = 7units. - Visual height ratio =
7 / 2 = 3.5x—making a modest5%gain look 3.5 times larger!
- Plan A bar height =
Corrected Matplotlib OOP Code:
import matplotlib.pyplot as plt
plans = ["Plan A", "Plan B"]
users = [100, 105]
fig, ax = plt.subplots(figsize=(5, 4))
bars = ax.bar(plans, users, color=["#8c8c8c", "#1f77b4"])
ax.set_ylim(0, 120)
ax.bar_label(bars, padding=3)
ax.set_title("Active Users: Plan B Leads Plan A by 5%")
ax.set_ylabel("Users")The true increase is 5.0% (1.05x), but starting the bar y-axis at 98 inflates the visual height ratio to 3.5x (7 / 2). Setting ax.set_ylim(0, 120) and adding ax.bar_label(bars) restores honest visual comparison.
5.3
How should a student answer an exam question asking: "Design an end-to-end Python visualization and reporting workflow using Matplotlib, Seaborn, and Plotly, specifying what each library contributes"?
Assigning Clear Roles to Each Library:
- Pandas Aggregation: Clean data and compute KPI summary tables (
groupby,.resample). - Matplotlib (
fig, ax = plt.subplots): Produce crisp, publication-grade multi-panel overview figures (png/svg) with custom annotations, consistent axes, and high DPI. - Seaborn (
boxplot,heatmap,pairplot): Reveal statistical distributions, outliers, and correlation structure across subgroups with minimal code. - Plotly (
plotly.express+write_html): Deliver interactive stakeholder drill-down charts with hover tooltips, legend filtering, and zoom saved as self-contained HTML.
Adding Storytelling and Delivery Structure: Conclude with the inverted-pyramid layout (KPIs → visual insights with action-oriented titles → drill-down links) and automated export into an HTML/PDF weekly report.
Describe a 4-stage workflow: Pandas aggregation → Matplotlib OOP multi-panel KPI charts → Seaborn statistical diagnostics (boxplot, heatmap) → Plotly interactive HTML drill-downs, unified under an inverted-pyramid narrative with insight-driven titles.
Exercises
Practice Exercises — Data Visualization and Storytelling
Explain Edward Tufte's data-ink ratio and state which chart type is most appropriate for: (a) tracking monthly electricity consumption over 3 years, (b) comparing median salary and outliers across 5 departments, and (c) inspecting pairwise correlations among 8 numeric features.
Data-Ink Ratio: The data-ink ratio is the proportion of a graphic's ink devoted to displaying non-redundant data information (data-ink / total-ink). Removing 3D bars, heavy gridlines, decorative frames, and redundant legends increases clarity and comprehension speed.
Choosing Appropriate Charts:
- (a) Monthly consumption over 3 years: Line chart (emphasizes temporal continuity and trend).
- (b) Median salary and outliers across 5 departments: Box plot (or violin + box overlay; displays median, IQR spread, skew, and outliers simultaneously).
- (c) Pairwise correlations among 8 numeric features: Annotated Heatmap (
sns.heatmap(df.corr(), annot=True, center=0)).
The data-ink ratio measures the share of visual ink representing actual data rather than clutter. Use a line chart for (a), a box plot for (b), and an annotated correlation heatmap for (c).
Why is Matplotlib's object-oriented API (fig, ax = plt.subplots(...)) preferred over the stateful plt.plot(...) API for data science projects? Write a 2x1 subplot example plotting a line chart on top and a histogram below with sharex=False.
Why OOP (fig, ax) Beats Stateful plt: Global plt functions operate implicitly on whichever figure/axes happens to be "current," causing bugs in multi-panel layouts, functions, and automated batch scripts. Explicit Figure and Axes objects let you target axes[0], axes[1] deterministically and pass ax= into Seaborn or helper functions.
2x1 Matplotlib OOP Subplot Example:
import matplotlib.pyplot as plt
fig, axes = plt.subplots(2, 1, figsize=(7, 6), constrained_layout=True)
axes[0].plot([1, 2, 3, 4], [10, 14, 13, 18], marker="o")
axes[0].set_title("Weekly Trend")
axes[1].hist([12, 15, 15, 18, 20, 22, 22, 25], bins=5, edgecolor="black")
axes[1].set_title("Score Distribution")The OOP API binds every plot call to explicit fig and ax objects, preventing state collisions in multi-panel and automated scripts. Use fig, axes = plt.subplots(2, 1) and call axes[0].plot(...) and axes[1].hist(...).
Compare Matplotlib, Seaborn, and Plotly across abstraction level, interactivity, and ideal use cases. Give one code snippet using plotly.express to build an interactive scatter plot saved to HTML.
Library Comparison:
- Matplotlib: Low-level static engine; total control over publication figures, layouts, and custom annotations.
- Seaborn: High-level statistical wrapper around Matplotlib working directly on tidy Pandas DataFrames (
hue,boxplot,pairplot,heatmap). - Plotly: Interactive browser-ready charts (
plotly.express/graph_objects) with hover tooltips, zoom/pan, legend toggling, and standalone HTML export.
Plotly Express Snippet:
import plotly.express as px
fig = px.scatter(df, x="hours_studied", y="exam_score", color="section",
hover_data=["student_id"], title="Study Hours vs. Exam Score")
fig.write_html("study_scatter.html")Matplotlib offers low-level static customization; Seaborn offers concise DataFrame-based statistical plots on Matplotlib; Plotly offers interactive browser charts (zoom, hover, HTML export via fig.write_html()).
What insights can you read from a Seaborn pairplot and a correlation heatmap? For a DataFrame with 8 numeric columns, how many total subplots appear in a full sns.pairplot(df) grid, and how many unique off-diagonal pairs exist?
Reading Pair Plots and Heatmaps:
- A pair plot reveals univariate distributions on the diagonal (skewness, multimodality) and pairwise non-linear shapes, clusters, or class separation on off-diagonal scatter plots.
- A correlation heatmap summarizes linear (or Spearman) strength and sign (
-1to+1) at a glance, quickly flagging multicollinear feature pairs (|r| > 0.85) and strong predictors of the target.
Grid Size Calculation for 8 Columns:
- Full
pairplotgrid dimensions =8 x 8 = 64subplots (8 diagonal distribution plots + 56 off-diagonal scatter plots). - Number of unique off-diagonal pairs =
8 * (8 - 1) / 2 = 28pairs (whichcorner=Truedisplays without mirroring duplicates).
Pair plots reveal diagonal distributions, non-linear relationships, and cluster separation, while heatmaps quantify pairwise correlation strength and multicollinearity. For 8 numeric columns, a full pairplot renders 64 subplots (8 x 8) representing 28 unique pairs.
Explain the three pillars of data storytelling and the inverted-pyramid dashboard layout. If weekly support tickets dropped from 600 to 480 after deploying a self-service FAQ, calculate the percentage change and rewrite the generic chart title "Support Tickets by Week" into an insight-driven headline.
Data Storytelling Pillars and Dashboard Layout:
- Three Pillars: (1) Verified Data, (2) Focused Visuals (minimal clutter, strategic color/annotations), and (3) Narrative (Context → Finding → Action).
- Inverted Pyramid: Top = KPI summary cards; Middle = trend and breakdown charts; Bottom = granular filterable tables.
Calculating Percentage Change and Rewriting the Title:
- Percentage change =
((480 - 600) / 600) * 100 = (-120 / 600) * 100 = -20.0%(a 20% reduction). - Insight-driven title:
"Self-Service FAQ Cut Weekly Support Tickets by 20% (600 to 480)".
Data storytelling combines verified data, focused visuals, and an actionable narrative, while dashboards follow an inverted pyramid (KPIs → trends/breakdowns → tables). Tickets fell by 20.0%, yielding the headline: "Self-Service FAQ Cut Weekly Support Tickets by 20% (600 to 480)".