INT303 Big Data Analysis - Coding Project 2: Loan Approval Prediction
Weightage: 100 points(30%of total course grade)
Due Date:12 DEC
Submission: Submit your Jupyter Notebook(.ipynb) and a concise 1-2 page executive summary report via [Learning Mall/Submission Portal].
1.Introduction
In the world of finance,accurate and efficient loan approval decisions are
paramount.Banks and financial institutions rely on robust data analysis and
predictive models to assess applicant creditworthiness,mitigate risks,and optimize their lending portfolios.This project challenges you to step into the role of a Data Scientist at a burgeoning financial technology(FinTech)firm.Your task is to
develop a machine learning model that predicts whether a loan application will be approved or rejected based on a comprehensive set of applicant data.
This project aims to solidify your understanding of the entire machine learning pipeline,from exploratory data analysis and preprocessing to model building, evaluation,and interpretation.You will be provided with a dataset containing various applicant attributes and their corresponding loan approval status.
2.Project Objectives
Upon completion of this project,you should be able to:
Perform comprehensive Exploratory Data Analysis(EDA)to understand data distributions,identify potential issues,and derive insights.
Implement effective data preprocessing techniques,including handling missing values,encoding categorical features,and scaling numerical features.
Engineer new,meaningful features from existing ones to enhance model performance.
Select and implement appropriate machine learning models for classification tasks.
· Evaluate model performance using various metrics and techniques.
· Interpret model results and explain the factors influencing loan approval decisions.
Present your findings clearly and professionally in a technical report.
Demonstrate proficiency in Python programming for data analysis and machine learning.
3.Dataset
You will be working with a dataset named loan_approval_dataset_copy.csv(a
sample of the "architsharma01/loan-approval-prediction-dataset").This dataset contains the following columns:
loan_id:Unique identifier for each loan application.
· no_of_dependents:Number of dependents the applicant has.
· education:Applicant's education level(Graduate/Not Graduate).
· self_employed:Whether the applicant is self-employed (Yes/No).
· income_annum:Applicant's annual income.
loan_amount:The requested loan amount.
loan_term:The duration of the loan in years.
· cibil_score:Applicant's CIBIL credit score(a creditworthiness indicator).
· residential_assets_value:Value of residential assets.
commercial_assets_value:Value of commercial assets.
luxury_assets_value:Value of luxury assets.
· bank_asset_value:Value of bank assets.
loan_status:The target variable,indicating whether the loan was 'Approved' or 'Rejected'.
Note: The provided CSV is a small sample.Assume you are working with a larger, more realistic version of this dataset where you may encounter missing values,
outliers,and varying data distributions.Your solution should be scalable and robust enough to handle such real-world scenarios.
4.Project Tasks
Your submission should include a well-commented Jupyter Notebook and a
separate executive summary report(PDF)summarizing your approach,findings, and recommendations.
Task 1:Exploratory Data Analysis(EDA)and Data Preprocessing(30 points)
1. Load and Initial Inspection: Load the dataset into a Pandas DataFrame. Display the first few rows,check data types,and identify missing values. Summarize key statistics.
2. Univariate Analysis: Analyze the distribution of each feature.For numerical features,create histograms and box plots.For categorical features,create bar plots.Describe your observations.
3. Bivariate Analysis: Explore the relationships between features,particularly their relationship with the loan_status target variable. Use appropriate visualizations(e.g.,scatter plots,stacked bar plots,heatmaps).
4. Data Cleaning: Handle any identified missing values,outliers(if present),or inconsistencies.Justify your chosen methods.
5. Feature Engineering: Create at least two new,meaningful features that you believe could improve model performance.Explain your rationale.
6. Categorical Encoding: Convert all categorical features into numerical representations suitable for machine learning models (e.g.,One-Hot Encoding, Label Encoding).
Feature Scaling: Apply appropriate scaling techniques(e.g.,StandardScaler, MinMaxScaler)to numerical features.
Task 2:Model Development and Evaluation(40 points)
Data Splitting: Split your processed data into training and testing sets(e.g., 70%training,30%testing).
Model Selection: Choose at least three different classification algorithms.
Good candidates might include:
■ Logistic Regression
Decision Tree Classifier
Random Forest Classifier
Gradient Boosting Classifier(e.g.,XGBoost,LightGBM) Support Vector Machine(SVM)
K-Nearest Neighbors(KNN)
Model Training: Train your chosen models on the training data.
Hyperparameter Tuning: Implement a strategy to tune hyperparameters for each selected model (e.g.,GridSearchCV,RandomizedSearchCV).Explain why hyperparameter tuning is important.
Model Evaluation: Evaluate the performance of each tuned model on the test set using various metrics.At a minimum,include:
Accuracy
Precision,Recall,F1-score (for both 'Approved'and 'Rejected'classes) ROC AUC Score
Confusion Matrix
Provide a comparative analysis of the models based on these metrics, considering the business context(e.g.,what kind of errors are more costly for a bank?).
6. Feature Importance (if applicable): For tree-based models,analyze and visualize feature importance.Discuss which features your model deems most crucial for loan approval prediction.
Task 3:Executive Summary Report(20 points)
Write a 1-2 page executive summary report (in PDF format)that addresses the following:
1. Introduction: Briefly state the problem and the objective of your project.
2. Methodology: Summarize your data preprocessing steps,feature engineering choices,and the models you experimented with.
3. Key Findings: Present the performance of your best models using relevant metrics.Discuss the most important features.
4. Recommendations &Insights: Based on your analysis,what insights can you provide to the FinTech firm regarding loan approval?Which model would you recommend and why?Suggest potential improvements or next steps for future work.
5. Ethical Considerations(Bonus,5 points):Briefly discuss any ethical considerations related to building and deploying such a loan approval model(e.g., bias,fairness,transparency).
Task 4:Code Quality and Documentation (10 points)
1. Code Readability: Your Jupyter Notebook should be well-structured,logical, and easy to follow.
2. Comments: Include appropriate comments to explain complex logic,choices, and reasoning.
3. Reproducibility: Ensure your notebook can be run from top to bottom without errors and produces consistent results.