Data Scientist& AI Engineer
Prabhath Vipparthi
Building ML systems
that work in production.
I'm a Data Scientist and AI Engineer with 4 years of experience building production ML systems across enterprise and AI-native environments.
Currently at Scale AI as an Applied AI Engineer — I prepare training datasets, engineer ML features, build classification models, and implement embedding-based retrieval workflows that reduce manual review effort by 25%.
Before Scale AI, I spent over 3 years at HCL Tech building churn prediction and demand forecasting models for enterprise clients — improving high-risk recall by 18% and cutting forecast error by approximately 15%.
I'm completing my MS in Data Science at NJIT (GPA 3.7, May 2026) and actively looking for my next full-time role.
ML & Modeling
- Classification & regression (scikit-learn, PyTorch, TensorFlow)
- Cross-validation, precision/recall tuning, ROC-AUC
- Feature engineering, SMOTE, SHAP explainability
NLP & AI Systems
- Transformers, Hugging Face, spaCy, embeddings
- RAG pipelines with pgvector and similarity search
- LLM evaluation, LLM-as-a-Judge, vLLM inference
Data Engineering
- PySpark, Apache Spark, Databricks at scale
- dbt, Apache Airflow, DuckDB, medallion architecture
- ETL pipelines, data quality tests, Apache Arrow
MLOps & Deployment
- MLflow experiment tracking and model registry
- Docker, Kubernetes, Azure, AWS, Render
- Model monitoring, data drift detection, CI/CD
Three things I care about,
learned the hard way.
Ship what runs.
Production over prototype. A model that hits 0.94 in a notebook but drifts in the first week isn't shipped — it's abandoned. I care about the version that survives real data, real users, and real Mondays.
Explain the model, not just its score.
SHAP values, decision logs, audit trails. Stakeholders don't sign off on ROC-AUC — they sign off on "why did this customer get flagged." Every classifier I ship comes with a plain-English answer to that question.
Test the pipeline, not just the notebook.
ML fails silently. Wrong join, drifted feature, stale schema — the model still returns a number. 700+ automated tests across my projects because catching it at 3am with a paged alert is worse than catching it in CI.
Featured projects.
AI Digital Badge Classification — NJIT LDI Capstone
Production AI-assisted classification system automating NJIT's institutional taxonomy across 3 dimensions, validated on 20 real-world held-out submissions with 100% accuracy. Built a 4-layer text extraction pipeline — 130+ lexicon phrase patterns, 44 regex rules, spaCy Bloom's taxonomy matcher, and an LLM stub — handling OBv3 JSON, guided forms, and free-text inputs. Implemented a deterministic 3-stage rule engine (19 rules) with immutable audit logs, plain-English decision explanations, and human-in-the-loop override workflows.
FinSight — Pre-Market Intelligence Platform
Production pre-market briefing system that runs every weekday morning. Ingests ~130 symbols, computes RSI and moving-average indicators, generates commentary with Gemini 2.5 Flash, stores everything in Postgres with pgvector, and emails confirmed subscribers via Resend. A retrieval-augmented Q&A layer answers plain-English questions over the full briefing history with inline date citations.
StarCoder2 Self-Alignment Pipeline
Implemented the SelfOSSInstruct methodology from the StarCoder2 paper to generate a TypeScript instruction-tuning dataset. Extracted functions from The Stack v2 using tree-sitter AST parsing and TypeScript compiler type-checking, then ran an S→C→I→R chain (Seed → Concepts → Instructions → Responses) via StarCoder2-3B on a T4 GPU. Produced 448 complete instruction-response pairs from 5,791 type-checked seeds. Filtered final outputs with model-based quality scoring.
NYC Taxi Medallion Data Pipeline
End-to-end data engineering pipeline over 5.97M real NYC TLC Yellow Taxi trips (Jan–Feb 2024): raw Parquet → PySpark cleaning → dbt gold marts → 23 data-quality tests → daily Airflow DAG. Built on a DuckDB warehouse with a Streamlit dashboard layer. Analysis surfaced Manhattan as 75% of total revenue ($111.9M of $149.1M).
Other projects.
Olist SQL Analysis & Hypothesis Testing
End-to-end SQL analysis on real Olist Brazilian e-commerce data (99,441 orders across 96,096 customers). Six DuckDB window-function queries power a monthly cohort retention matrix. A formal power analysis and two-proportion z-test confirmed a payment-method retention hypothesis as null (p = 0.76) with 80% statistical power on a 0.34 pp minimum detectable effect.
Loan Approval Risk Prediction
End-to-end classification pipeline on 20,000 loan applications (36 features, 76.1% rejection baseline). Applied SMOTE on training folds only to avoid data leakage. Compared 6 models — Logistic Regression, Decision Tree, Random Forest, SVM, KNN, and ANN — with GridSearchCV. SHAP identified CreditScore, AnnualIncome, and DebtToIncomeRatio as top predictors, packaged for compliance-ready reporting.
Cryptocurrency Market Analysis on Hadoop
Three distributed MapReduce jobs in Java analyzing 2 GB of historical OHLCV tick data across 100+ cryptocurrency pairs (Binance, Apr–Aug 2024) on a multi-node AWS EC2 Hadoop cluster. Jobs surface volatility rankings, worst-performing assets by open-to-close change, and cumulative volume leaders with peak timestamps.
User Management System
FastAPI + PostgreSQL backend with JWT OAuth2 authentication, role-based access control (Admin, Manager, User), and profile-completion tracking. Diagnosed and resolved 5 critical production bugs across CI failures, unique-constraint violations, routing 404s, nested transaction errors, and mocking issues. Added 10 edge-case tests (138 passing) and shipped full CI/CD with GitHub Actions and Docker.
Where I’ve worked.
Scale AI
Current- Built Python, SQL, and PySpark pipelines processing 10K+ records, cutting manual validation effort by 30%.
- Prepared training and evaluation datasets and engineered statistical and ML features (Pandas, NumPy, Scikit-learn) for automated anomaly and data-quality detection.
- Built classification models (Scikit-learn, PyTorch) with cross-validation, precision/recall analysis, and threshold tuning to improve minority-class detection.
- Integrated Hugging Face transformers and embeddings for text classification and semantic evaluation, reducing manual review activity by 25%.
- Implemented embedding-based retrieval workflows and Python REST APIs for similarity search and AI/LLM evaluation.
- Supported model deployment using MLflow, Docker, Kubernetes, Azure, and Databricks.
- Monitored model quality, data drift, latency, and pipeline failures; optimized PySpark workloads reducing processing time by 28%.
- Created Power BI dashboards tracking AI/ML performance, data-quality trends, and business KPIs.
HCL Tech
- Partnered with enterprise business stakeholders to translate retention and growth objectives into churn and forecasting use cases.
- Conducted EDA and statistical analysis (Python, Pandas, NumPy, SQL) to uncover customer behavior patterns and key churn drivers.
- Engineered 30+ customer-level features (purchase frequency, recency, transaction value, engagement) for predictive modeling.
- Built and compared Logistic Regression, Random Forest, and Gradient Boosting classifiers with cross-validation and ROC-AUC, improving high-risk customer recall by 18%.
- Developed demand forecasting models reducing forecast error by ~15%, supporting operational planning.
- Processed 5K+ transaction records with PySpark and Databricks, cutting recurring batch runtime by 35%.
- Automated data and model workflows using Airflow, AWS, and MLflow; delivered Tableau dashboards covering churn segments, forecasts, and KPIs.
Master of Science in Data Science
Everything I actually use.
Credentials & Training
AI Fluency for Builders
Covers how to design and build production-ready systems using Claude — prompt engineering, tool use, multi-step pipeline…
Introduction to Agent Skills
Hands-on course covering agentic AI design patterns, tool-calling, memory management, and orchestrating multi-step Claud…
Supervised Machine Learning: Regression and Classification
Covers linear and logistic regression, gradient descent, regularization, and neural network fundamentals — the core curr…
Let's work together.
I'm actively looking for full-time roles as a Data Scientist, AI Engineer, ML Engineer, or Data Engineer. Onsite, hybrid, or remote — anywhere in the US.