Past exam question: 2081 Chaitra (Regular), Q7(iii)
Open in the Padhaiko appWrite brief notes on: (Any Three)
(iii) Generative AI
Solution
Definition and Paradigmatic Shift of Generative AI: Generative Artificial Intelligence (GenAI) refers to advanced deep learning models that generate realistic, novel content—including natural language text, synthetic tabular data, realistic imagery, audio, and executable computer code—by learning the underlying joint probability distribution of large unstructured training corpora.
This represents a major paradigm shift from traditional Discriminative AI:
- Discriminative Models: Learn the conditional boundary to classify or predict labels for given inputs (e.g., spam vs. non-spam, disease diagnosis).
- Generative Models: Learn the joint distribution or the data distribution to synthesize realistic new samples that resemble the training distribution but are not verbatim copies.
Foundational Model Architectures: Generative AI relies on several transformative neural architectures:
- Transformer Architecture & Self-Attention: The engine behind modern Large Language Models (LLMs like GPT, LLaMA, Claude, Gemini). Self-attention mechanisms enable the model to capture long-range contextual dependencies across tokens simultaneously rather than sequentially.
- Diffusion Models: Learn to reconstruct high-fidelity data by reversing a progressive forward noising process (gradually destroying data with Gaussian noise, then training a neural network to denoise it step-by-step; powers tools like Stable Diffusion and Midjourney).
- Generative Adversarial Networks (GANs): Formulated as a two-player zero-sum game between a Generator network that creates synthetic candidates and a Discriminator network that attempts to distinguish real training samples from generated fakes.
- Variational Autoencoders (VAEs): Map high-dimensional inputs to a continuous, probabilistic latent space, allowing new instances to be synthesized by sampling latent coordinates.
Applications and Practical Challenges in Data Science:
- Data Science Applications:
- Synthetic Data Generation: Augmenting sparse datasets and addressing class imbalance while safeguarding privacy.
- Automated Code & Query Synthesis: Accelerating data preparation through LLM-assisted SQL querying, pandas script generation, and documentation.
- Multimodal Analysis: Processing joint image-text-data workflows for automated exploratory reporting.
- Key Challenges & Ethical Risks:
- Hallucinations: Generating convincing yet factually false or nonsensical statements.
- Algorithmic Bias: Amplifying toxic, discriminatory, or historical prejudices present in web-scale pre-training data.
- Compute and Energy Intensity: High GPU infrastructure requirements for training and serving foundation models.
Generative AI comprises deep learning architectures (Transformers, Diffusion models, GANs, VAEs) that learn data distributions to generate novel text, images, code, and synthetic data, shifting machine learning from classification to synthesis. In data science, GenAI powers synthetic data generation, automated coding, and multimodal analysis, while presenting operational challenges around hallucinations, algorithmic bias, and computational costs.