PromptBase
🤖 GPT
This advanced meta-prompt template guides developers through the process of designing and implementing a custom backend using Node.js. By specifying key parameters such as project requirements, architectural decisions, API design, database integration, security measures, and deployment strategies, this template helps generate a comprehensive plan for backend development.
...more
$6.99
After purchasing, you will gain access to the prompt file which you can use with GPT or the app builder. You'll receive 20 free generation credits with this purchase. By purchasing this prompt, you agree to our terms of service.
Over 1 month ago
Related item
Prompt Details
Model
Chat - (o1-mini)
Token size
640 ($0.02496 / call)
Example input
[project_name]: EduConnect: Adaptive Learning Backend
[project_requirements]: User authentication, course management, progress tracking, adaptive learning algorithms, video streaming integration, quiz system
[architecture_design]: Monolithic architecture with modular components for future microservices transition
[api_specifications]: RESTful API for user management, course CRUD, progress tracking; GraphQL API for complex data queries; WebSocket for real-time updates
[database_strategy]: PostgreSQL for relational data (users, courses, quizzes); MongoDB for unstructured data (learning materials, user progress)
[authentication_method]: JWT authentication with refresh tokens; OAuth2 for social media login
[security_measures]: HTTPS, bcrypt for password hashing, XSS protection, CSRF tokens, rate limiting
[performance_considerations]: Redis caching for frequently accessed course data, database query optimization, CDN for static assets
[testing_approach]: Jest for unit testing, Supertest for API testing, Cypress for E2E testing, GitHub Actions for CI/CD
[deployment_plan]: Docker containers, AWS ECS for orchestration, Blue-Green deployment strategy
[scalability_strategy]: Horizontal scaling of application servers, database read replicas, content delivery network for global reach
[monitoring_logging]: AWS CloudWatch for logging, Datadog for application performance monitoring, PagerDuty for alerting
[documentation_standards]: Swagger for API documentation, JSDoc for code documentation, README.md files for each module, Confluence for system architecture
Example output
### Step 1: Project Initialization and Setup
Let's begin by setting up the foundational aspects of the **EduConnect: Adaptive Learning Backend** project. This step ensures that your development environment is properly configured and that the project structure is organized for scalability and maintainability.
---
#### 1. **Initialize the Node.js Project**
**a. Install Node.js and npm**
- Ensure that you have [Node.js](https://nodejs.org/) and npm installed on your machine.
- Verify the installation by running:
```bash
node -v
npm -v
```
**b. Create Project Directory**
```bash
mkdir educonnect-backend
cd educonnect-backend
```
**c. Initialize npm**
```bash
npm init -y
```
This command creates a `package.json` file with default settings.
---
#### 2. **Set Up Version Control**
**a. Initialize Git Repository**
```bash
git init
```
**b. Create `.gitignore` File**
Create a `.gitignore` file to exclude unnecessary files from version control.
```bash
touch .gitignore
```
**Add the following to `.gitignore`:**
```
node_modules/
.env
dist/
coverage/
```
**c. Initial Commit**
```bash
git add .
git commit -m "Initial commit: Project setup and structure"
```
---
#### 3. **Define Project Structure**
Organizing your project into well-defined directories enhances maintainability and scalability. Here's a recommended structure based on your project requirements:
```
educonnect-backend/
├── src/
│ ├── controllers/
│ ├── models/
│ ├── routes/
│ ├── services/
│ ├── utils/
│ ├── middleware/
│ ├── config/
│ └── index.js
├── tests/
│ ├── unit/
│ ├── integration/
│ └── e2e/
├── config/
│ ├── default.json
│ └── production.json
├── scripts/
├── .env
├── .gitignore
├── package.json
├── README.md
└── Dockerfile
```
**Directory Breakdown:**
- `src/`: Contains the main application code.
- `controllers/`: Handles incoming requests and responses.
- `models/`: Defines the data schemas and interacts with the database.
- `routes/`: Defines API endpoints.
- `services/`: Contains business logic and interacts with external services.
- `utils/`: Utility functions and helpers.
- `middleware/`: Custom middleware for request processing.
- `config/`: Configuration files and environment variables.
- `index.js`: Entry point of the application.
- `tests/`: Contains all test cases.
- `unit/`: Unit tests.
- `integration/`: Integration tests.
- `e2e/`: End-to-end tests.
- `config/`: Configuration files for different environments.
- `scripts/`: Deployment and build scripts.
- `.env`: Environment variables (ensure this is added to `.gitignore`).
- `Dockerfile`: For containerization.
**Create the Directory Structure:**
```bash
mkdir -p src/{controllers,models,routes,services,utils,middleware,config}
mkdir -p tests/{unit,integration,e2e}
mkdir config scripts
touch src/index.js
touch README.md
touch Dockerfile
```
---
#### 4. **Set Up Development Environment**
**a. Install Essential Dependencies**
```bash
npm install express dotenv cors
```
- `express`: Web framework for Node.js.
- `dotenv`: Loads environment variables from a `.env` file.
- `cors`: Enables Cross-Origin Resource Sharing.
**b. Install Development Dependencies**
```bash
npm install --save-dev nodemon eslint prettier
```
- `nodemon`: Automatically restarts the server on file changes.
- `eslint`: Linting tool for identifying and fixing code issues.
- `prettier`: Code formatter.
**c. Configure `package.json` Scripts**
Update the `scripts` section in `package.json`:
```json
"scripts": {
"start": "node src/index.js",
"dev": "nodemon src/index.js",
"lint": "eslint .",
"format": "prettier --write .",
"test": "jest"
},
```
**d. Initialize ESLint**
```bash
npx eslint --init
```
- Follow the prompts to set up ESLint according to your coding standards.
**e. Initialize Prettier**
Create a `.prettierrc` file for Prettier configuration:
```json
{
"semi": true,
"singleQuote": true,
"tabWidth": 2,
"trailingComma": "es5"
}
```
---
#### 5. **Set Up Environment Variables**
**a. Create `.env` File**
```bash
touch .env
```
**Add the following variables:**
```
PORT=3000
DATABASE_URL=postgres://username:password@localhost:5432/educonnect
JWT_SECRET=your_jwt_secret_key
```
**b. Load Environment Variables in `src/index.js`**
```javascript
require('dotenv').config();
const express = require('express');
const app = express();
const PORT = process.env.PORT || 3000;
app.use(express.json());
app.get('/', (req, res) => {
res.send('EduConnect Backend is Running');
});
app.listen(PORT, () => {
console.log(`Server is running on port ${PORT}`);
});
```
---
#### 6. **Containerization with Docker (Optional but Recommended)**
**a. Create `Dockerfile`**
```dockerfile
# Use Node.js LTS version
FROM node:18
# Create app directory
WORKDIR /usr/src/app
# Install app dependencies
COPY package*.json ./
RUN npm install
# Bundle app source
COPY . .
# Expose port
EXPOSE 3000
# Start the app
CMD ["npm", "start"]
```
**b. Create `.dockerignore`**
```bash
node_modules
npm-debug.log
```
---
#### 7. **Initial Commit and Push to Repository**
**a. Create a Remote Repository**
- Create a repository on GitHub, GitLab, or any other platform of your choice.
**b. Add Remote and Push**
```bash
git remote add origin https://github.com/yourusername/educonnect-backend.git
git branch -M main
git push -u origin main
```
---
#### 8. **Set Up Continuous Integration (CI) with GitHub Actions (Optional at This Stage)**
You can set up CI pipelines in later steps, but initializing the structure now can be beneficial.
---
### **Summary**
In **Step 1**, we've successfully:
- **Initialized** a new Node.js project with `npm`.
- **Set up version control** using Git, including a `.gitignore` file.
- **Defined a scalable project structure** tailored to your project requirements.
- **Configured the development environment** with essential dependencies and tools like ESLint and Prettier.
- **Set up environment variables** to manage configuration securely.
- **Prepared for containerization** with a `Dockerfile`.
- **Performed the initial commit** and pushed the project to a remote repository.
---
### **Next Steps**
With the project initialized and the development environment configured, we're ready to move on to **Step 2: Define Architecture and API Design**. This will involve outlining the monolithic architecture with modular components and designing the API endpoints as specified.
Feel free to let me know if you need any clarifications or assistance with this step!
===
### Step 2: Define Architecture and API Design
In **Step 2**, we'll outline the overall architecture for **EduConnect: Adaptive Learning Backend** and design the API endpoints based on your specifications. This step ensures that the system is well-structured, scalable, and aligns with your project requirements.
---
#### 1. **Outline the Overall Architecture**
Given your requirement for a **Monolithic architecture with modular components for future microservices transition**, we'll design the backend to be a single unified application divided into distinct modules. This modular approach facilitates easier maintenance, scalability, and a smoother transition to microservices if needed in the future.
**Monolithic Architecture with Modular Components:**
- **Monolithic Core:** The entire application runs as a single process, simplifying deployment and development initially.
- **Modular Separation:** Each functional area (e.g., User Management, Course Management, Progress Tracking) is encapsulated within its own module.
- **Loose Coupling:** Modules interact through well-defined interfaces, reducing dependencies and enhancing maintainability.
- **Future-Proofing:** The architecture allows individual modules to be extracted into microservices with minimal impact on the overall system.
**Architecture Diagram:**
```
+-----------------------------------------------------------+
| EduConnect Backend |
| |
| +-----------+ +-----------+ +-----------+ |
| | User | | Course | | Progress | |
| | Module | | Module | | Module | |
| +-----------+ +-----------+ +-----------+ |
| | | | |
| +-------+-------+-------+-------+ |
| | | |
| +----+----+ +---+---+ |
| | RESTful | | GraphQL| |
| | API | | API | |
| +---------+ +-------+ |
| | | |
| +-----+-----+ | |
| | WebSocket | | |
| +-----------+ | |
| | | |
| +------+-------+ | |
| | Database | <-----+ |
| | (PostgreSQL, | |
| | MongoDB) | |
| +---------------+ |
+-----------------------------------------------------------+
```
**Key Components:**
- **User Module:** Handles user authentication, registration, profile management.
- **Course Module:** Manages course creation, updating, deletion, and retrieval.
- **Progress Module:** Tracks user progress, quiz results, and adaptive learning data.
- **RESTful API:** Exposes endpoints for user management, course CRUD operations, and progress tracking.
- **GraphQL API:** Facilitates complex data queries, allowing clients to request exactly the data they need.
- **WebSocket:** Enables real-time updates such as live notifications, progress updates, and collaborative features.
- **Database Layer:** Utilizes PostgreSQL for relational data and MongoDB for unstructured data.
---
#### 2. **Design API Endpoints and Data Flow**
Based on your **API specifications**, we'll design the RESTful API, GraphQL API, and WebSocket endpoints. Here's a detailed breakdown:
##### **a. RESTful API Design**
The RESTful API will handle standard CRUD operations and user management functionalities.
**Base URL:** `https://api.educonnect.com/v1`
**Endpoints:**
1. **User Management**
- **Register User**
- `POST /users/register`
- **Description:** Register a new user.
- **Request Body:**
```json
{
"username": "john_doe",
"email": "john@example.com",
"password": "SecureP@ssw0rd"
}
```
- **Response:**
```json
{
"id": "uuid",
"username": "john_doe",
"email": "john@example.com",
"token": "JWT_TOKEN"
}
```
- **Login User**
- `POST /users/login`
- **Description:** Authenticate a user and issue a JWT.
- **Request Body:**
```json
{
"email": "john@example.com",
"password": "SecureP@ssw0rd"
}
```
- **Response:**
```json
{
"token": "JWT_TOKEN",
"refreshToken": "REFRESH_TOKEN"
}
```
- **Get User Profile**
- `GET /users/profile`
- **Description:** Retrieve authenticated user's profile.
- **Headers:** `Authorization: Bearer JWT_TOKEN`
- **Response:**
```json
{
"id": "uuid",
"username": "john_doe",
"email": "john@example.com",
"roles": ["student"]
}
```
- **Update User Profile**
- `PUT /users/profile`
- **Description:** Update authenticated user's profile.
- **Headers:** `Authorization: Bearer JWT_TOKEN`
- **Request Body:** (Any updatable fields)
```json
{
"username": "john_doe_updated",
"email": "john_new@example.com"
}
```
- **Response:** Updated user profile.
- **Delete User**
- `DELETE /users/profile`
- **Description:** Delete authenticated user's account.
- **Headers:** `Authorization: Bearer JWT_TOKEN`
- **Response:** Confirmation message.
2. **Course Management**
- **Create Course**
- `POST /courses`
- **Description:** Create a new course.
- **Headers:** `Authorization: Bearer JWT_TOKEN` (Admin role)
- **Request Body:**
```json
{
"title": "Introduction to Node.js",
"description": "Learn the basics of Node.js.",
"category": "Programming",
"videos": ["video_url_1", "video_url_2"],
"materials": ["material_url_1", "material_url_2"]
}
```
- **Response:** Created course details.
- **Get All Courses**
- `GET /courses`
- **Description:** Retrieve a list of all courses.
- **Response:** Array of courses.
- **Get Course by ID**
- `GET /courses/:courseId`
- **Description:** Retrieve details of a specific course.
- **Response:** Course details.
- **Update Course**
- `PUT /courses/:courseId`
- **Description:** Update course details.
- **Headers:** `Authorization: Bearer JWT_TOKEN` (Admin role)
- **Request Body:** Fields to update.
- **Response:** Updated course details.
- **Delete Course**
- `DELETE /courses/:courseId`
- **Description:** Delete a course.
- **Headers:** `Authorization: Bearer JWT_TOKEN` (Admin role)
- **Response:** Confirmation message.
3. **Progress Tracking**
- **Get User Progress**
- `GET /progress`
- **Description:** Retrieve the authenticated user's progress.
- **Headers:** `Authorization: Bearer JWT_TOKEN`
- **Response:** Progress data.
- **Update User Progress**
- `PUT /progress`
- **Description:** Update the authenticated user's progress.
- **Headers:** `Authorization: Bearer JWT_TOKEN`
- **Request Body:**
```json
{
"courseId": "uuid",
"completedModules": ["module1", "module2"],
"quizScores": {
"quiz1": 85,
"quiz2": 90
}
}
```
- **Response:** Updated progress data.
4. **Quizzes**
- **Create Quiz**
- `POST /quizzes`
- **Description:** Create a new quiz.
- **Headers:** `Authorization: Bearer JWT_TOKEN` (Instructor/Admin role)
- **Request Body:**
```json
{
"courseId": "uuid",
"questions": [
{
"questionText": "What is Node.js?",
"options": ["Option1", "Option2", "Option3", "Option4"],
"correctAnswer": "Option1"
},
// More questions
]
}
```
- **Response:** Created quiz details.
- **Get Quiz by ID**
- `GET /quizzes/:quizId`
- **Description:** Retrieve quiz details.
- **Response:** Quiz data.
- **Submit Quiz Answers**
- `POST /quizzes/:quizId/submit`
- **Description:** Submit answers for a quiz.
- **Headers:** `Authorization: Bearer JWT_TOKEN`
- **Request Body:**
```json
{
"answers": {
"question1": "Option1",
"question2": "Option3",
// More answers
}
}
```
- **Response:** Quiz results and score.
##### **b. GraphQL API Design**
The GraphQL API will handle complex data queries that may require fetching related data across multiple modules. This allows clients to request precisely the data they need in a single request.
**Endpoint:** `https://api.educonnect.com/graphql`
**Schema Overview:**
```graphql
type User {
id: ID!
username: String!
email: String!
roles: [String!]!
progress: Progress
}
type Course {
id: ID!
title: String!
description: String!
category: String!
videos: [String!]!
materials: [String!]!
quizzes: [Quiz!]
}
type Quiz {
id: ID!
courseId: ID!
questions: [Question!]!
}
type Question {
id: ID!
questionText: String!
options: [String!]!
correctAnswer: String!
}
type Progress {
courseId: ID!
completedModules: [String!]!
quizScores: [QuizScore!]
}
type QuizScore {
quizId: ID!
score: Int!
}
type Query {
user(id: ID!): User
course(id: ID!): Course
courses(category: String, search: String): [Course!]
quiz(id: ID!): Quiz
progress(userId: ID!, courseId: ID!): Progress
}
type Mutation {
createUser(username: String!, email: String!, password: String!): User
updateUser(id: ID!, username: String, email: String): User
deleteUser(id: ID!): Boolean
createCourse(title: String!, description: String!, category: String!): Course
updateCourse(id: ID!, title: String, description: String, category: String): Course
deleteCourse(id: ID!): Boolean
createQuiz(courseId: ID!, questions: [QuestionInput!]!): Quiz
submitQuiz(quizId: ID!, answers: [AnswerInput!]!): QuizResult
}
input QuestionInput {
questionText: String!
options: [String!]!
correctAnswer: String!
}
input AnswerInput {
questionId: ID!
selectedOption: String!
}
type QuizResult {
quizId: ID!
score: Int!
total: Int!
correctAnswers: [CorrectAnswer!]
}
type CorrectAnswer {
questionId: ID!
correctOption: String!
}
```
**Sample Queries and Mutations:**
- **Fetch a Course with Quizzes:**
```graphql
query {
course(id: "course_uuid") {
id
title
description
quizzes {
id
questions {
id
questionText
options
}
}
}
}
```
- **Fetch User Progress:**
```graphql
query {
progress(userId: "user_uuid", courseId: "course_uuid") {
completedModules
quizScores {
quizId
score
}
}
}
```
- **Submit Quiz Answers:**
```graphql
mutation {
submitQuiz(quizId: "quiz_uuid", answers: [
{ questionId: "q1", selectedOption: "Option1" },
{ questionId: "q2", selectedOption: "Option3" }
]) {
quizId
score
total
correctAnswers {
questionId
correctOption
}
}
}
```
##### **c. WebSocket Design**
WebSockets will facilitate real-time communication between the server and clients. This is useful for features like live notifications, real-time progress updates, and collaborative learning.
**WebSocket Endpoint:** `wss://api.educonnect.com/ws`
**Use Cases:**
1. **Real-Time Notifications:**
- **Event:** New course available, course updates, quiz availability.
- **Message Format:**
```json
{
"event": "NEW_COURSE",
"data": {
"courseId": "uuid",
"title": "New Course Title"
}
}
```
2. **Progress Updates:**
- **Event:** User completes a module or quiz.
- **Message Format:**
```json
{
"event": "PROGRESS_UPDATE",
"data": {
"userId": "uuid",
"courseId": "uuid",
"completedModules": ["module1", "module2"],
"quizScores": {
"quiz1": 85,
"quiz2": 90
}
}
}
```
3. **Live Collaborative Features:**
- **Event:** Real-time chat in study groups or live Q&A sessions.
- **Message Format:**
```json
{
"event": "CHAT_MESSAGE",
"data": {
"userId": "uuid",
"message": "Hello, I have a question about Node.js!"
}
}
```
**Implementation Considerations:**
- **Authentication:** Use JWT tokens to authenticate WebSocket connections.
- **Scalability:** Integrate with a message broker (e.g., Redis Pub/Sub) if scaling horizontally.
- **Handling Events:** Define a clear protocol for event types and data structures to ensure consistency.
---
#### 3. **Data Flow Overview**
Understanding how data flows through the system is crucial for designing efficient and reliable interactions between different components.
**Typical Data Flow Scenarios:**
1. **User Registration:**
- **Client:** Sends `POST /users/register` with user details.
- **Server:** Validates input, hashes password, stores user in PostgreSQL, issues JWT.
- **Response:** Returns user data with JWT.
- **WebSocket:** Optionally, send a welcome notification.
2. **Course Creation:**
- **Admin Client:** Sends `POST /courses` with course details.
- **Server:** Validates input, stores course in PostgreSQL and/or MongoDB, notifies via WebSocket.
- **Response:** Returns created course data.
- **WebSocket:** Broadcast `NEW_COURSE` event to subscribed clients.
3. **Fetching Courses:**
- **Client:** Sends `GET /courses` or a GraphQL query.
- **Server:** Retrieves courses from PostgreSQL/MongoDB, applies caching with Redis if applicable.
- **Response:** Returns list of courses.
4. **Tracking Progress:**
- **Client:** Sends `PUT /progress` with progress data.
- **Server:** Updates progress in MongoDB, recalculates adaptive learning metrics.
- **Response:** Returns updated progress data.
- **WebSocket:** Sends `PROGRESS_UPDATE` event to the user.
5. **Submitting Quiz Answers:**
- **Client:** Sends `POST /quizzes/:quizId/submit` with answers.
- **Server:** Validates answers, calculates score, updates progress.
- **Response:** Returns quiz results and score.
- **WebSocket:** Notifies user of quiz completion and score.
6. **Real-Time Notifications:**
- **Server:** Upon certain events (e.g., new course), sends messages via WebSocket.
- **Client:** Receives and handles notifications in real-time.
**Data Flow Diagram:**
```
Client
|
|-- HTTP Requests (REST/GraphQL) --> Server Backend
| |
| |-- Database Operations (PostgreSQL, MongoDB)
| |
| |-- Business Logic (Controllers, Services)
| |
| |-- Cache Operations (Redis)
|
|-- WebSocket Messages <--> Server Backend
```
---
#### 4. **Modular Component Breakdown**
To ensure a clean separation of concerns and facilitate future microservices migration, we'll define distinct modules within the monolithic architecture. Each module encapsulates related functionalities.
**Modules:**
1. **User Module**
- **Responsibilities:** User registration, authentication, profile management, role assignments.
- **Components:**
- **Controllers:** Handle HTTP requests related to users.
- **Models:** Define user schemas for PostgreSQL.
- **Routes:** Define user-related API endpoints.
- **Services:** Implement business logic for user operations.
- **Middleware:** Authentication and authorization checks.
2. **Course Module**
- **Responsibilities:** Course creation, management, retrieval.
- **Components:**
- **Controllers:** Handle HTTP requests related to courses.
- **Models:** Define course schemas for PostgreSQL/MongoDB.
- **Routes:** Define course-related API endpoints.
- **Services:** Implement business logic for course operations.
3. **Progress Module**
- **Responsibilities:** Tracking user progress, adaptive learning data.
- **Components:**
- **Controllers:** Handle HTTP requests related to progress tracking.
- **Models:** Define progress schemas for MongoDB.
- **Routes:** Define progress-related API endpoints.
- **Services:** Implement business logic for progress tracking.
4. **Quiz Module**
- **Responsibilities:** Quiz creation, management, submission.
- **Components:**
- **Controllers:** Handle HTTP requests related to quizzes.
- **Models:** Define quiz schemas for PostgreSQL/MongoDB.
- **Routes:** Define quiz-related API endpoints.
- **Services:** Implement business logic for quiz operations.
5. **Notification Module**
- **Responsibilities:** Real-time notifications via WebSockets.
- **Components:**
- **WebSocket Handlers:** Manage WebSocket connections and events.
- **Services:** Implement notification dispatch logic.
6. **Common Utilities**
- **Responsibilities:** Shared functionalities across modules.
- **Components:**
- **Authentication Middleware:** Verify JWT tokens.
- **Error Handling:** Centralized error management.
- **Validation:** Input validation for requests.
**Example Directory Structure with Modules:**
```
src/
├── controllers/
│ ├── userController.js
│ ├── courseController.js
│ ├── progressController.js
│ └── quizController.js
├── models/
│ ├── User.js
│ ├── Course.js
│ ├── Progress.js
│ └── Quiz.js
├── routes/
│ ├── userRoutes.js
│ ├── courseRoutes.js
│ ├── progressRoutes.js
│ └── quizRoutes.js
├── services/
│ ├── userService.js
│ ├── courseService.js
│ ├── progressService.js
│ └── quizService.js
├── middleware/
│ ├── authMiddleware.js
│ ├── errorHandler.js
│ └── validationMiddleware.js
├── utils/
│ ├── logger.js
│ └── helpers.js
├── config/
│ ├── db.js
│ └── websocket.js
├── graphql/
│ ├── schema.js
│ └── resolvers.js
├── websocket/
│ └── websocketHandler.js
├── index.js
└── app.js
```
---
#### 5. **Technology Stack Alignment**
Ensure that the chosen technologies and libraries support the architectural and API design decisions.
- **Express.js:** Core framework for building RESTful APIs.
- **Apollo Server:** For implementing the GraphQL API within the Express server.
- **Socket.io or ws:** For managing WebSocket connections and real-time communication.
- **Sequelize or TypeORM:** ORM for PostgreSQL to handle relational data.
- **Mongoose:** ODM for MongoDB to handle unstructured data.
- **GraphQL Tools:** To define schemas and resolvers effectively.
- **Authentication Libraries:** `jsonwebtoken` for JWT handling, `passport` for OAuth2 integration.
---
#### 6. **API Documentation**
Maintain comprehensive documentation for both RESTful and GraphQL APIs to facilitate development and onboarding.
- **Swagger (OpenAPI):** Document RESTful endpoints.
- **GraphQL Playground:** Provide an interactive environment for exploring the GraphQL API.
**Setting Up Swagger:**
1. **Install Swagger Dependencies:**
```bash
npm install swagger-ui-express swagger-jsdoc
```
2. **Configure Swagger in `src/index.js`:**
```javascript
const swaggerUi = require('swagger-ui-express');
const swaggerJsdoc = require('swagger-jsdoc');
const options = {
definition: {
openapi: '3.0.0',
info: {
title: 'EduConnect API',
version: '1.0.0',
description: 'API documentation for EduConnect Backend',
},
servers: [
{
url: 'https://api.educonnect.com/v1',
},
],
},
apis: ['./src/routes/*.js'], // Path to the API docs
};
const swaggerSpec = swaggerJsdoc(options);
app.use('/api-docs', swaggerUi.serve, swaggerUi.setup(swaggerSpec));
```
3. **Annotate Routes for Swagger:**
```javascript
// src/routes/userRoutes.js
/**
* @swagger
* /users/register:
* post:
* summary: Register a new user
* tags: [Users]
* requestBody:
* required: true
* content:
* application/json:
* schema:
* type: object
* required:
* - username
* - email
* - password
* properties:
* username:
* type: string
* email:
* type: string
* password:
* type: string
* responses:
* 201:
* description: User created successfully
* 400:
* description: Bad request
*/
router.post('/register', userController.register);
```
**Accessing Swagger UI:**
Navigate to `https://api.educonnect.com/v1/api-docs` to view the interactive API documentation.
---
#### 7. **API Versioning Strategy**
Implement API versioning to manage changes and ensure backward compatibility.
- **URL-Based Versioning:** Include the version number in the URL path.
- Example: `https://api.educonnect.com/v1/users/register`
- **Implementation:**
- Organize routes under versioned directories.
- Maintain separate controllers and services if breaking changes are introduced in future versions.
**Example Directory Structure with Versioning:**
```
src/
├── routes/
│ ├── v1/
│ │ ├── userRoutes.js
│ │ ├── courseRoutes.js
│ │ └── ...
│ └── v2/
│ ├── userRoutes.js
│ ├── courseRoutes.js
│ └── ...
```
**Registering Versioned Routes in `index.js`:**
```javascript
const userRoutesV1 = require('./routes/v1/userRoutes');
const courseRoutesV1 = require('./routes/v1/courseRoutes');
// Import other v1 routes
app.use('/v1/users', userRoutesV1);
app.use('/v1/courses', courseRoutesV1);
// Use other v1 routes
```
---
#### 8. **Error Handling Strategy**
Establish a consistent and comprehensive error handling mechanism to provide meaningful feedback to clients and facilitate debugging.
**Centralized Error Handler Middleware:**
1. **Create `errorHandler.js` in `middleware/`:**
```javascript
// src/middleware/errorHandler.js
module.exports = (err, req, res, next) => {
console.error(err.stack);
res.status(err.status || 500).json({
success: false,
message: err.message || 'Internal Server Error',
});
};
```
2. **Use Error Handler in `index.js`:**
```javascript
const errorHandler = require('./middleware/errorHandler');
// ... other middlewares and routes
// Error handling middleware should be the last middleware
app.use(errorHandler);
```
**Creating Custom Error Classes:**
1. **Define Custom Errors:**
```javascript
// src/utils/CustomError.js
class CustomError extends Error {
constructor(message, status) {
super(message);
this.status = status;
}
}
module.exports = CustomError;
```
2. **Use Custom Errors in Controllers:**
```javascript
// src/controllers/userController.js
const CustomError = require('../utils/CustomError');
exports.register = async (req, res, next) => {
try {
// Registration logic
if (userExists) {
throw new CustomError('User already exists', 400);
}
// ...
} catch (error) {
next(error);
}
};
```
---
#### 9. **API Rate Limiting and Throttling**
Implement rate limiting to protect the API from abuse and ensure fair usage.
**Using `express-rate-limit`:**
1. **Install the Package:**
```bash
npm install express-rate-limit
```
2. **Configure Rate Limiting Middleware:**
```javascript
// src/middleware/rateLimiter.js
const rateLimit = require('express-rate-limit');
const limiter = rateLimit({
windowMs: 15 * 60 * 1000, // 15 minutes
max: 100, // limit each IP to 100 requests per windowMs
message: 'Too many requests from this IP, please try again after 15 minutes',
headers: true,
});
module.exports = limiter;
```
3. **Apply Rate Limiting in `index.js`:**
```javascript
const rateLimiter = require('./middleware/rateLimiter');
app.use('/api', rateLimiter); // Apply to all API routes
```
---
#### 10. **CORS Configuration**
Configure Cross-Origin Resource Sharing (CORS) to control which domains can access your API.
**Using `cors` Package:**
1. **Install CORS:**
```bash
npm install cors
```
2. **Configure CORS in `index.js`:**
```javascript
const cors = require('cors');
const corsOptions = {
origin: ['https://www.educonnect.com', 'https://app.educonnect.com'], // Allowed origins
methods: ['GET', 'POST', 'PUT', 'DELETE'],
allowedHeaders: ['Content-Type', 'Authorization'],
credentials: true,
};
app.use(cors(corsOptions));
```
**Handling Preflight Requests:**
Ensure that your server correctly handles HTTP OPTIONS requests, which are used in preflight checks by browsers.
---
#### 11. **Logging Strategy**
Implement comprehensive logging to monitor application behavior, troubleshoot issues, and analyze usage patterns.
**Using `winston` for Logging:**
1. **Install Winston:**
```bash
npm install winston
```
2. **Configure Logger:**
```javascript
// src/utils/logger.js
const { createLogger, format, transports } = require('winston');
const logger = createLogger({
level: 'info',
format: format.combine(
format.timestamp({
format: 'YYYY-MM-DD HH:mm:ss',
}),
format.errors({ stack: true }),
format.splat(),
format.json()
),
defaultMeta: { service: 'educonnect-backend' },
transports: [
new transports.File({ filename: 'logs/error.log', level: 'error' }),
new transports.File({ filename: 'logs/combined.log' }),
],
});
// If in development, log to console as well
if (process.env.NODE_ENV !== 'production') {
logger.add(
new transports.Console({
format: format.combine(format.colorize(), format.simple()),
})
);
}
module.exports = logger;
```
3. **Use Logger in Application:**
```javascript
// src/index.js
const logger = require('./utils/logger');
app.listen(PORT, () => {
logger.info(`Server is running on port ${PORT}`);
});
// Example usage in controllers
// src/controllers/userController.js
const logger = require('../utils/logger');
exports.register = async (req, res, next) => {
try {
// Registration logic
logger.info(`User registered: ${user.email}`);
res.status(201).json({ /* ... */ });
} catch (error) {
logger.error('Registration error: %o', error);
next(error);
}
};
```
---
#### 12. **API Versioning and Documentation Integration**
Integrate API versioning and documentation seamlessly to provide a clear and organized API structure.
**Combining Versioning with Swagger:**
- Ensure that Swagger documentation includes versioned endpoints.
- Update Swagger configuration to reflect different API versions.
**Example Swagger Configuration for Multiple Versions:**
```javascript
const options = {
definition: {
openapi: '3.0.0',
info: {
title: 'EduConnect API',
version: '1.0.0',
description: 'API documentation for EduConnect Backend',
},
servers: [
{
url: 'https://api.educonnect.com/v1',
},
{
url: 'https://api.educonnect.com/v2',
},
],
},
apis: ['./src/routes/v1/*.js', './src/routes/v2/*.js'],
};
```
---
### **Summary**
In **Step 2**, we've:
- **Outlined the Monolithic Architecture:** Structured the backend into modular components, facilitating future scalability.
- **Designed RESTful, GraphQL, and WebSocket APIs:** Defined endpoints, schemas, and real-time communication channels to meet project requirements.
- **Established Data Flow and Modular Breakdown:** Ensured a clear separation of concerns and efficient data handling across different modules.
- **Aligned Technology Stack:** Selected appropriate libraries and tools to support the architectural and API design decisions.
- **Implemented API Documentation and Versioning:** Provided comprehensive documentation and a versioning strategy to maintain API clarity and compatibility.
- **Set Up Error Handling, Rate Limiting, CORS, and Logging:** Enhanced the robustness, security, and observability of the backend.
---
### **Next Steps**
With the architecture and API design defined, we're ready to proceed to **Step 3: Database Integration**. In this step, we'll set up the database solutions as per your strategy, configure connections, and establish ORM/ODM frameworks.
Feel free to ask any questions or request further clarifications on Step 2!
===
### Step 3: Database Integration
In **Step 3**, we'll set up the database solutions for **EduConnect: Adaptive Learning Backend** based on your **database_strategy**. This involves integrating **PostgreSQL** for relational data (users, courses, quizzes) and **MongoDB** for unstructured data (learning materials, user progress). We'll configure connections, set up ORMs/ODMs, and define schemas/models to interact with the databases effectively.
---
#### 1. **Choose and Configure Database Solutions**
Given your project requirements, we'll use:
- **PostgreSQL** for relational data:
- **Data Models:** Users, Courses, Quizzes
- **ORM:** **Sequelize** (a promise-based Node.js ORM for PostgreSQL)
- **MongoDB** for unstructured data:
- **Data Models:** Learning Materials, User Progress
- **ODM:** **Mongoose** (an elegant MongoDB object modeling tool)
**Why These Choices?**
- **Sequelize:** Provides a robust ORM with support for migrations, associations, and easy query building for PostgreSQL.
- **Mongoose:** Offers a straightforward ODM with schema validation, middleware, and seamless integration with MongoDB.
---
#### 2. **Install Necessary Dependencies**
Navigate to your project directory and install the required packages:
```bash
# PostgreSQL and Sequelize
npm install sequelize pg pg-hstore
# Sequelize CLI for migrations and model generation
npm install --save-dev sequelize-cli
# MongoDB and Mongoose
npm install mongoose
```
**Explanation of Packages:**
- `sequelize`: The core Sequelize library.
- `pg`: PostgreSQL client for Node.js.
- `pg-hstore`: A module for serializing and deserializing JSON data into the hstore format used by PostgreSQL.
- `sequelize-cli`: Command-line interface for Sequelize, useful for migrations and model management.
- `mongoose`: ODM for MongoDB.
---
#### 3. **Configure PostgreSQL with Sequelize**
##### a. **Initialize Sequelize**
Initialize Sequelize in your project to set up the necessary configuration files.
```bash
npx sequelize-cli init
```
This command creates the following folders:
```
config/
└── config.json
models/
migrations/
seeders/
```
##### b. **Update Sequelize Configuration**
Modify the `config/config.json` file to include your PostgreSQL database credentials. It's recommended to use environment variables for sensitive information.
**Example `config/config.json`:**
```json
{
"development": {
"username": "your_pg_username",
"password": "your_pg_password",
"database": "educonnect_dev",
"host": "127.0.0.1",
"dialect": "postgres"
},
"test": {
"username": "your_pg_username",
"password": "your_pg_password",
"database": "educonnect_test",
"host": "127.0.0.1",
"dialect": "postgres"
},
"production": {
"username": "your_pg_username",
"password": "your_pg_password",
"database": "educonnect_prod",
"host": "127.0.0.1",
"dialect": "postgres"
}
}
```
**Using Environment Variables:**
To enhance security, replace hardcoded credentials with environment variables. Install the `dotenv` package if not already installed.
```bash
npm install dotenv
```
**Update `config/config.js`:**
Rename `config/config.json` to `config/config.js` and modify it to use environment variables.
```javascript
// config/config.js
require('dotenv').config(); // Load environment variables
module.exports = {
development: {
username: process.env.PG_USERNAME,
password: process.env.PG_PASSWORD,
database: process.env.PG_DATABASE_DEV,
host: process.env.PG_HOST,
dialect: 'postgres'
},
test: {
username: process.env.PG_USERNAME,
password: process.env.PG_PASSWORD,
database: process.env.PG_DATABASE_TEST,
host: process.env.PG_HOST,
dialect: 'postgres'
},
production: {
username: process.env.PG_USERNAME,
password: process.env.PG_PASSWORD,
database: process.env.PG_DATABASE_PROD,
host: process.env.PG_HOST,
dialect: 'postgres'
}
};
```
**Update Environment Variables in `.env`:**
```env
# PostgreSQL Configuration
PG_USERNAME=your_pg_username
PG_PASSWORD=your_pg_password
PG_DATABASE_DEV=educonnect_dev
PG_DATABASE_TEST=educonnect_test
PG_DATABASE_PROD=educonnect_prod
PG_HOST=127.0.0.1
```
##### c. **Define Sequelize Models**
Create models for **User**, **Course**, and **Quiz**.
**Example: User Model**
```bash
npx sequelize-cli model:generate --name User --attributes username:string,email:string,password:string,roles:array
```
This command generates a model and a migration file for the `User` model. Modify the generated files as needed.
**`models/user.js`:**
```javascript
// models/user.js
'use strict';
const { Model } = require('sequelize');
module.exports = (sequelize, DataTypes) => {
class User extends Model {
static associate(models) {
// Define associations here
User.hasMany(models.Course, { foreignKey: 'instructorId', as: 'courses' });
User.hasMany(models.Progress, { foreignKey: 'userId', as: 'progresses' });
// Add more associations as needed
}
}
User.init(
{
username: {
type: DataTypes.STRING,
allowNull: false,
unique: true
},
email: {
type: DataTypes.STRING,
allowNull: false,
unique: true,
validate: {
isEmail: true
}
},
password: {
type: DataTypes.STRING,
allowNull: false
},
roles: {
type: DataTypes.ARRAY(DataTypes.STRING),
defaultValue: ['student']
}
},
{
sequelize,
modelName: 'User',
tableName: 'Users',
timestamps: true
}
);
return User;
};
```
**Example: Course Model**
```bash
npx sequelize-cli model:generate --name Course --attributes title:string,description:text,category:string,instructorId:uuid
```
**`models/course.js`:**
```javascript
// models/course.js
'use strict';
const { Model } = require('sequelize');
module.exports = (sequelize, DataTypes) => {
class Course extends Model {
static associate(models) {
// Define associations here
Course.belongsTo(models.User, { foreignKey: 'instructorId', as: 'instructor' });
Course.hasMany(models.Quiz, { foreignKey: 'courseId', as: 'quizzes' });
// Add more associations as needed
}
}
Course.init(
{
title: {
type: DataTypes.STRING,
allowNull: false
},
description: {
type: DataTypes.TEXT,
allowNull: false
},
category: {
type: DataTypes.STRING,
allowNull: false
},
instructorId: {
type: DataTypes.UUID,
allowNull: false,
references: {
model: 'Users',
key: 'id'
}
}
},
{
sequelize,
modelName: 'Course',
tableName: 'Courses',
timestamps: true
}
);
return Course;
};
```
**Example: Quiz Model**
```bash
npx sequelize-cli model:generate --name Quiz --attributes title:string,courseId:uuid,questions:jsonb
```
**`models/quiz.js`:**
```javascript
// models/quiz.js
'use strict';
const { Model } = require('sequelize');
module.exports = (sequelize, DataTypes) => {
class Quiz extends Model {
static associate(models) {
// Define associations here
Quiz.belongsTo(models.Course, { foreignKey: 'courseId', as: 'course' });
}
}
Quiz.init(
{
title: {
type: DataTypes.STRING,
allowNull: false
},
courseId: {
type: DataTypes.UUID,
allowNull: false,
references: {
model: 'Courses',
key: 'id'
}
},
questions: {
type: DataTypes.JSONB,
allowNull: false
}
},
{
sequelize,
modelName: 'Quiz',
tableName: 'Quizzes',
timestamps: true
}
);
return Quiz;
};
```
**Run Migrations:**
After defining the models, run the migrations to create the corresponding tables in PostgreSQL.
```bash
npx sequelize-cli db:create
npx sequelize-cli db:migrate
```
**Notes:**
- Ensure that PostgreSQL is running and accessible with the provided credentials.
- You can create seeders if you need to populate the database with initial data.
---
#### 4. **Configure MongoDB with Mongoose**
##### a. **Set Up MongoDB Connection**
Create a configuration file for MongoDB connection using Mongoose.
**Create `config/mongoose.js`:**
```javascript
// config/mongoose.js
const mongoose = require('mongoose');
const logger = require('../src/utils/logger'); // Assuming you have a logger utility
const connectDB = async () => {
try {
await mongoose.connect(process.env.MONGO_URI, {
useNewUrlParser: true,
useUnifiedTopology: true,
// useFindAndModify: false, // Deprecated in Mongoose 6
// useCreateIndex: true // Deprecated in Mongoose 6
});
logger.info('MongoDB connected successfully');
} catch (error) {
logger.error('MongoDB connection error:', error);
process.exit(1); // Exit process with failure
}
};
module.exports = connectDB;
```
**Update `.env` with MongoDB URI:**
```env
# MongoDB Configuration
MONGO_URI=mongodb://username:password@localhost:27017/educonnect
```
##### b. **Define Mongoose Schemas and Models**
Create schemas for **Learning Materials** and **User Progress**.
**Example: Learning Material Schema**
```javascript
// models/mongoose/LearningMaterial.js
const mongoose = require('mongoose');
const LearningMaterialSchema = new mongoose.Schema(
{
title: {
type: String,
required: true
},
content: {
type: String, // Could also be rich text or markdown
required: true
},
courseId: {
type: String, // Reference to Course ID in PostgreSQL
required: true
},
resourceType: {
type: String,
enum: ['video', 'document', 'image', 'other'],
default: 'other'
},
url: {
type: String,
required: true
}
},
{ timestamps: true }
);
module.exports = mongoose.model('LearningMaterial', LearningMaterialSchema);
```
**Example: User Progress Schema**
```javascript
// models/mongoose/UserProgress.js
const mongoose = require('mongoose');
const UserProgressSchema = new mongoose.Schema(
{
userId: {
type: String, // Reference to User ID in PostgreSQL
required: true
},
courseId: {
type: String, // Reference to Course ID in PostgreSQL
required: true
},
completedModules: {
type: [String],
default: []
},
quizScores: {
type: Map,
of: Number, // Map of quizId to score
default: {}
},
adaptiveData: {
// Data used by adaptive learning algorithms
type: Object,
default: {}
}
},
{ timestamps: true }
);
module.exports = mongoose.model('UserProgress', UserProgressSchema);
```
##### c. **Connect to MongoDB in the Application Entry Point**
Modify your `src/index.js` to connect to both PostgreSQL and MongoDB during startup.
**`src/index.js`:**
```javascript
// src/index.js
require('dotenv').config();
const express = require('express');
const app = express();
const http = require('http');
const httpServer = http.createServer(app);
// PostgreSQL Connection
const { sequelize } = require('./models'); // Sequelize instance
// MongoDB Connection
const connectDB = require('../config/mongoose');
// Middleware Setup
app.use(express.json());
// Import Routes
const userRoutes = require('./routes/v1/userRoutes');
// ... import other routes
// Use Routes
app.use('/v1/users', userRoutes);
// ... use other routes
// API Documentation
const swaggerUi = require('swagger-ui-express');
const swaggerJsdoc = require('swagger-jsdoc');
const swaggerOptions = {
definition: {
openapi: '3.0.0',
info: {
title: 'EduConnect API',
version: '1.0.0',
description: 'API documentation for EduConnect Backend',
},
servers: [
{
url: 'https://api.educonnect.com/v1',
},
],
},
apis: ['./src/routes/v1/*.js'], // Path to the API docs
};
const swaggerSpec = swaggerJsdoc(swaggerOptions);
app.use('/api-docs', swaggerUi.serve, swaggerUi.setup(swaggerSpec));
// Error Handling Middleware
const errorHandler = require('./middleware/errorHandler');
app.use(errorHandler);
// Connect to Databases and Start Server
const PORT = process.env.PORT || 3000;
const startServer = async () => {
try {
// Connect to PostgreSQL
await sequelize.authenticate();
console.log('PostgreSQL connected successfully.');
// Connect to MongoDB
await connectDB();
// Start the server
httpServer.listen(PORT, () => {
console.log(`Server is running on port ${PORT}`);
});
} catch (error) {
console.error('Database connection failed:', error);
process.exit(1);
}
};
startServer();
```
**Explanation:**
- **Sequelize Connection:** Uses the `sequelize.authenticate()` method to verify the connection to PostgreSQL.
- **Mongoose Connection:** Invokes the `connectDB` function to establish a connection to MongoDB.
- **Error Handling:** If either connection fails, the server exits to prevent running without a database connection.
---
#### 5. **Define Relationships and Associations**
Properly defining relationships ensures data integrity and simplifies data retrieval.
##### a. **PostgreSQL Associations with Sequelize**
**User and Course Relationship:**
- **One-to-Many:** A **User** (instructor) can create multiple **Courses**.
**`models/user.js`:**
```javascript
// models/user.js
// ... existing code
static associate(models) {
// One-to-Many: User -> Courses
User.hasMany(models.Course, { foreignKey: 'instructorId', as: 'courses' });
// Additional associations
User.hasMany(models.Progress, { foreignKey: 'userId', as: 'progresses' });
}
```
**Course and Quiz Relationship:**
- **One-to-Many:** A **Course** can have multiple **Quizzes**.
**`models/course.js`:**
```javascript
// models/course.js
// ... existing code
static associate(models) {
// Belongs to User
Course.belongsTo(models.User, { foreignKey: 'instructorId', as: 'instructor' });
// One-to-Many: Course -> Quizzes
Course.hasMany(models.Quiz, { foreignKey: 'courseId', as: 'quizzes' });
}
```
**Quiz Association:**
- **Belongs to Course**
**`models/quiz.js`:**
```javascript
// models/quiz.js
// ... existing code
static associate(models) {
// Belongs to Course
Quiz.belongsTo(models.Course, { foreignKey: 'courseId', as: 'course' });
}
```
##### b. **MongoDB References**
Since MongoDB is used for unstructured data, references to PostgreSQL entities (like `courseId` and `userId`) are maintained as simple string fields. This denormalized approach is suitable for unstructured data and allows for flexibility.
---
#### 6. **Implement Repository or Service Layers**
To maintain a clean separation of concerns and facilitate easier testing and maintenance, implement repository or service layers that handle database interactions.
**Example: User Service for PostgreSQL**
```javascript
// src/services/userService.js
const { User } = require('../models');
const createUser = async (userData) => {
return await User.create(userData);
};
const findUserByEmail = async (email) => {
return await User.findOne({ where: { email } });
};
const findUserById = async (id) => {
return await User.findByPk(id, {
include: [{ model: Course, as: 'courses' }]
});
};
// Add more user-related database operations as needed
module.exports = {
createUser,
findUserByEmail,
findUserById,
// ... other exports
};
```
**Example: Learning Material Service for MongoDB**
```javascript
// src/services/learningMaterialService.js
const LearningMaterial = require('../models/mongoose/LearningMaterial');
const createLearningMaterial = async (materialData) => {
const material = new LearningMaterial(materialData);
return await material.save();
};
const getMaterialsByCourseId = async (courseId) => {
return await LearningMaterial.find({ courseId });
};
// Add more learning material-related database operations as needed
module.exports = {
createLearningMaterial,
getMaterialsByCourseId,
// ... other exports
};
```
**Benefits:**
- **Encapsulation:** Database logic is encapsulated within service layers, keeping controllers clean.
- **Reusability:** Services can be reused across different parts of the application.
- **Testability:** Services can be easily mocked or stubbed during testing.
---
#### 7. **Handle Migrations and Seeders for PostgreSQL**
Properly managing database schema changes is crucial for maintaining data integrity.
##### a. **Creating Migrations**
Sequelize CLI allows you to create migrations that modify the database schema.
**Example: Adding a New Column to Users Table**
```bash
npx sequelize-cli migration:generate --name add-profile-picture-to-users
```
**`migrations/XXXXXX-add-profile-picture-to-users.js`:**
```javascript
'use strict';
module.exports = {
up: async (queryInterface, Sequelize) => {
await queryInterface.addColumn('Users', 'profilePicture', {
type: Sequelize.STRING,
allowNull: true,
defaultValue: null
});
},
down: async (queryInterface, Sequelize) => {
await queryInterface.removeColumn('Users', 'profilePicture');
}
};
```
**Run Migrations:**
```bash
npx sequelize-cli db:migrate
```
##### b. **Creating Seeders**
Seeders allow you to populate the database with initial or test data.
**Example: Seeding Admin User**
```bash
npx sequelize-cli seed:generate --name admin-user
```
**`seeders/XXXXXX-admin-user.js`:**
```javascript
'use strict';
const bcrypt = require('bcrypt');
module.exports = {
up: async (queryInterface, Sequelize) => {
const hashedPassword = await bcrypt.hash('AdminP@ssw0rd', 10);
return queryInterface.bulkInsert('Users', [
{
id: 'admin-uuid',
username: 'admin',
email: 'admin@educonnect.com',
password: hashedPassword,
roles: ['admin'],
createdAt: new Date(),
updatedAt: new Date()
}
]);
},
down: async (queryInterface, Sequelize) => {
return queryInterface.bulkDelete('Users', { email: 'admin@educonnect.com' }, {});
}
};
```
**Run Seeders:**
```bash
npx sequelize-cli db:seed:all
```
**Notes:**
- Ensure that seeders are idempotent and can be safely rolled back if needed.
- Use UUIDs or other unique identifiers as necessary.
---
#### 8. **Implement Data Validation and Sanitization**
Ensuring data integrity is essential for both PostgreSQL and MongoDB. Use validation at the model level and within request handling.
##### a. **Sequelize Model Validations**
Leverage Sequelize's built-in validators.
**Example: User Model Email Validation**
```javascript
// models/user.js
email: {
type: DataTypes.STRING,
allowNull: false,
unique: true,
validate: {
isEmail: {
msg: 'Must be a valid email address'
}
}
}
```
##### b. **Mongoose Schema Validations**
Use Mongoose's schema validation features.
**Example: Learning Material URL Validation**
```javascript
// models/mongoose/LearningMaterial.js
url: {
type: String,
required: true,
validate: {
validator: function(v) {
return /^https?:\/\/.+\..+/.test(v);
},
message: props => `${props.value} is not a valid URL!`
}
}
```
##### c. **Request Validation with Middleware**
Implement request validation using middleware like **Joi** or **express-validator**.
**Install `express-validator`:**
```bash
npm install express-validator
```
**Example: User Registration Validation**
```javascript
// src/routes/v1/userRoutes.js
const express = require('express');
const router = express.Router();
const { register } = require('../../controllers/userController');
const { body } = require('express-validator');
const validate = require('../../middleware/validate');
router.post(
'/register',
[
body('username').notEmpty().withMessage('Username is required'),
body('email').isEmail().withMessage('Valid email is required'),
body('password')
.isLength({ min: 6 })
.withMessage('Password must be at least 6 characters long')
],
validate, // Middleware to handle validation results
register
);
module.exports = router;
```
**Create `validate.js` Middleware:**
```javascript
// src/middleware/validate.js
const { validationResult } = require('express-validator');
const validate = (req, res, next) => {
const errors = validationResult(req);
if (!errors.isEmpty()) {
return res.status(400).json({
success: false,
errors: errors.array().map(err => err.msg)
});
}
next();
};
module.exports = validate;
```
---
#### 9. **Implement Connection Pooling and Optimize Database Performance**
Efficient database connections and query optimizations are vital for performance.
##### a. **Sequelize Connection Pool Configuration**
Adjust the Sequelize configuration to use connection pooling.
**Update `config/config.js`:**
```javascript
// config/config.js
// ... existing code
development: {
username: process.env.PG_USERNAME,
password: process.env.PG_PASSWORD,
database: process.env.PG_DATABASE_DEV,
host: process.env.PG_HOST,
dialect: 'postgres',
pool: {
max: 10, // Maximum number of connections
min: 0,
acquire: 30000,
idle: 10000
}
},
// ... other environments
```
##### b. **Indexing in PostgreSQL**
Ensure that frequently queried fields are indexed to speed up read operations.
**Example: Adding Index on Email Field**
Modify the migration file or create a new migration to add an index.
```bash
npx sequelize-cli migration:generate --name add-email-index-to-users
```
**`migrations/XXXXXX-add-email-index-to-users.js`:**
```javascript
'use strict';
module.exports = {
up: async (queryInterface, Sequelize) => {
await queryInterface.addIndex('Users', ['email'], {
unique: true,
name: 'users_email_idx'
});
},
down: async (queryInterface, Sequelize) => {
await queryInterface.removeIndex('Users', 'users_email_idx');
}
};
```
**Run Migrations:**
```bash
npx sequelize-cli db:migrate
```
##### c. **Mongoose Connection Options**
Optimize Mongoose connection settings for performance.
```javascript
// config/mongoose.js
const mongoose = require('mongoose');
const logger = require('../src/utils/logger'); // Assuming you have a logger utility
const connectDB = async () => {
try {
await mongoose.connect(process.env.MONGO_URI, {
useNewUrlParser: true,
useUnifiedTopology: true,
// Additional options for performance
poolSize: 10, // Maintain up to 10 socket connections
serverSelectionTimeoutMS: 5000 // Timeout after 5s instead of 30s
});
logger.info('MongoDB connected successfully');
} catch (error) {
logger.error('MongoDB connection error:', error);
process.exit(1); // Exit process with failure
}
};
module.exports = connectDB;
```
---
#### 10. **Implement Caching with Redis**
Enhance performance by caching frequently accessed data using **Redis**.
##### a. **Install Redis and Dependencies**
**Install Redis on Your Machine:**
Refer to the [official Redis installation guide](https://redis.io/topics/quickstart) for your operating system.
**Install Redis Client for Node.js:**
```bash
npm install redis
```
##### b. **Configure Redis Client**
Create a Redis client and set up caching mechanisms.
**Create `config/redis.js`:**
```javascript
// config/redis.js
const redis = require('redis');
const logger = require('../src/utils/logger'); // Assuming you have a logger utility
const redisClient = redis.createClient({
url: process.env.REDIS_URL || 'redis://localhost:6379'
});
redisClient.on('error', (err) => {
logger.error('Redis Client Error', err);
});
const connectRedis = async () => {
try {
await redisClient.connect();
logger.info('Redis connected successfully');
} catch (error) {
logger.error('Redis connection error:', error);
}
};
module.exports = { redisClient, connectRedis };
```
**Update `.env` with Redis URL:**
```env
# Redis Configuration
REDIS_URL=redis://localhost:6379
```
##### c. **Integrate Redis in Application Entry Point**
Modify `src/index.js` to connect to Redis.
```javascript
// src/index.js
// ... existing imports
// Redis Connection
const { connectRedis } = require('../config/redis');
// ... existing code
const startServer = async () => {
try {
// Connect to PostgreSQL
await sequelize.authenticate();
console.log('PostgreSQL connected successfully.');
// Connect to MongoDB
await connectDB();
// Connect to Redis
await connectRedis();
// Start the server
httpServer.listen(PORT, () => {
console.log(`Server is running on port ${PORT}`);
});
} catch (error) {
console.error('Database connection failed:', error);
process.exit(1);
}
};
```
##### d. **Implement Caching in Services**
**Example: Caching Courses Data**
```javascript
// src/services/courseService.js
const { Course } = require('../models');
const { redisClient } = require('../../config/redis');
const getAllCourses = async () => {
const cacheKey = 'all_courses';
// Check Redis Cache
const cachedCourses = await redisClient.get(cacheKey);
if (cachedCourses) {
return JSON.parse(cachedCourses);
}
// Fetch from Database
const courses = await Course.findAll();
// Cache the Result in Redis for 1 hour
await redisClient.setEx(cacheKey, 3600, JSON.stringify(courses));
return courses;
};
module.exports = {
getAllCourses,
// ... other exports
};
```
**Benefits:**
- **Reduced Latency:** Faster data retrieval for cached data.
- **Lower Database Load:** Decreases the number of read operations on the database.
- **Scalability:** Supports handling high traffic by offloading repetitive queries to Redis.
---
#### 11. **Implement Database Security Best Practices**
Ensuring the security of your databases is paramount. Follow these best practices:
##### a. **Use Environment Variables for Credentials**
Avoid hardcoding sensitive information. Store all database credentials in environment variables.
**Example `.env`:**
```env
# PostgreSQL
PG_USERNAME=your_pg_username
PG_PASSWORD=your_pg_password
PG_DATABASE_DEV=educonnect_dev
PG_DATABASE_TEST=educonnect_test
PG_DATABASE_PROD=educonnect_prod
PG_HOST=127.0.0.1
# MongoDB
MONGO_URI=mongodb://username:password@localhost:27017/educonnect
# Redis
REDIS_URL=redis://localhost:6379
```
##### b. **Restrict Database Access**
- **Network Security:** Ensure databases are not publicly accessible. Use firewall rules to allow access only from trusted IPs or internal networks.
- **User Privileges:** Grant the minimum necessary privileges to database users. For example, the application user should only have access to specific databases and operations.
##### c. **Encrypt Data in Transit and at Rest**
- **TLS/SSL:** Use encrypted connections for PostgreSQL and MongoDB to protect data in transit.
**PostgreSQL Example:**
Update `config/config.js` to include SSL settings if connecting to a remote PostgreSQL server.
```javascript
// config/config.js
development: {
username: process.env.PG_USERNAME,
password: process.env.PG_PASSWORD,
database: process.env.PG_DATABASE_DEV,
host: process.env.PG_HOST,
dialect: 'postgres',
dialectOptions: {
ssl: {
require: true,
rejectUnauthorized: false // Set to true in production with valid certificates
}
},
pool: {
max: 10,
min: 0,
acquire: 30000,
idle: 10000
}
},
// ... other environments
```
- **At Rest:** Use disk encryption for databases if supported by your hosting environment.
##### d. **Regular Backups**
Implement regular backups for both PostgreSQL and MongoDB to prevent data loss.
- **PostgreSQL:** Use `pg_dump` for backups.
```bash
pg_dump -U your_pg_username -h your_pg_host -F c -b -v -f educonnect_dev.backup educonnect_dev
```
- **MongoDB:** Use `mongodump` for backups.
```bash
mongodump --uri="mongodb://username:password@localhost:27017/educonnect" --out=/backup/educonnect/
```
##### e. **Monitor and Audit Database Access**
Use monitoring tools to track database access and detect suspicious activities.
- **PostgreSQL:** Enable logging for connections and queries.
- **MongoDB:** Use MongoDB's built-in auditing features or integrate with external monitoring tools.
---
#### 12. **Test Database Integration**
Ensure that both PostgreSQL and MongoDB integrations work as expected by writing and running tests.
##### a. **Set Up Test Databases**
Create separate databases for testing to prevent contamination of development or production data.
**Update `.env.test`:**
```env
# PostgreSQL Test Configuration
PG_USERNAME=your_pg_username
PG_PASSWORD=your_pg_password
PG_DATABASE_TEST=educonnect_test
PG_HOST=127.0.0.1
# MongoDB Test Configuration
MONGO_URI=mongodb://username:password@localhost:27017/educonnect_test
# Redis Test Configuration
REDIS_URL=redis://localhost:6379
```
##### b. **Configure Jest for Testing**
Ensure Jest uses the test databases by setting the `NODE_ENV` to `test`.
**Update `package.json`:**
```json
"scripts": {
"test": "NODE_ENV=test jest"
}
```
##### c. **Write Tests for Services**
**Example: Testing User Service**
```javascript
// tests/unit/userService.test.js
const { createUser, findUserByEmail } = require('../../src/services/userService');
const { sequelize, User } = require('../../src/models');
const bcrypt = require('bcrypt');
beforeAll(async () => {
await sequelize.sync({ force: true }); // Recreate database
});
afterAll(async () => {
await sequelize.close();
});
describe('User Service', () => {
it('should create a new user', async () => {
const userData = {
username: 'testuser',
email: 'testuser@example.com',
password: await bcrypt.hash('password123', 10),
roles: ['student']
};
const user = await createUser(userData);
expect(user).toHaveProperty('id');
expect(user.email).toBe('testuser@example.com');
});
it('should find a user by email', async () => {
const user = await findUserByEmail('testuser@example.com');
expect(user).not.toBeNull();
expect(user.username).toBe('testuser');
});
});
```
**Example: Testing Learning Material Service**
```javascript
// tests/unit/learningMaterialService.test.js
const { createLearningMaterial, getMaterialsByCourseId } = require('../../src/services/learningMaterialService');
const mongoose = require('mongoose');
beforeAll(async () => {
await mongoose.connect(process.env.MONGO_URI, {
useNewUrlParser: true,
useUnifiedTopology: true
});
});
afterAll(async () => {
await mongoose.connection.dropDatabase();
await mongoose.connection.close();
});
describe('Learning Material Service', () => {
it('should create a new learning material', async () => {
const materialData = {
title: 'Node.js Basics',
content: 'Introduction to Node.js',
courseId: 'course-uuid',
resourceType: 'video',
url: 'https://example.com/video.mp4'
};
const material = await createLearningMaterial(materialData);
expect(material).toHaveProperty('_id');
expect(material.title).toBe('Node.js Basics');
});
it('should retrieve materials by course ID', async () => {
const materials = await getMaterialsByCourseId('course-uuid');
expect(materials.length).toBe(1);
expect(materials[0].title).toBe('Node.js Basics');
});
});
```
**Run Tests:**
```bash
npm test
```
**Notes:**
- Ensure that test databases are properly isolated.
- Use mock data to prevent dependency on actual data.
- Consider using tools like **Factory Girl** or **Faker** for generating test data.
---
#### 13. **Implement Data Access Patterns**
Adopt data access patterns that promote scalability and maintainability.
##### a. **Repository Pattern**
Encapsulate data access logic within repository classes.
**Example: User Repository**
```javascript
// src/repositories/userRepository.js
const { User } = require('../models');
class UserRepository {
async create(userData) {
return await User.create(userData);
}
async findByEmail(email) {
return await User.findOne({ where: { email } });
}
async findById(id) {
return await User.findByPk(id, {
include: [{ model: Course, as: 'courses' }]
});
}
// Add more methods as needed
}
module.exports = new UserRepository();
```
**Benefits:**
- **Abstraction:** Hides the complexity of ORM/ODM operations.
- **Flexibility:** Allows easy switching of ORM/ODM without affecting business logic.
- **Testability:** Facilitates mocking repositories during testing.
---
#### 14. **Handle Transactions and Data Consistency**
Ensure data consistency across both PostgreSQL and MongoDB, especially when operations involve both databases.
##### a. **Sequelize Transactions**
Use Sequelize's transaction management to handle multiple related operations atomically.
**Example: Creating a Course with Initial Quiz**
```javascript
// src/services/courseService.js
const { sequelize, Course, Quiz } = require('../models');
const CustomError = require('../utils/CustomError');
const createCourseWithQuiz = async (courseData, quizData) => {
const transaction = await sequelize.transaction();
try {
const course = await Course.create(courseData, { transaction });
const quiz = await Quiz.create(
{ ...quizData, courseId: course.id },
{ transaction }
);
await transaction.commit();
return { course, quiz };
} catch (error) {
await transaction.rollback();
throw new CustomError('Failed to create course and quiz', 500);
}
};
module.exports = {
createCourseWithQuiz,
// ... other exports
};
```
**Notes:**
- Transactions ensure that either all operations succeed or none do, maintaining data integrity.
- Handle exceptions to rollback transactions in case of failures.
##### b. **Cross-Database Transactions**
Handling transactions that span both PostgreSQL and MongoDB is complex due to the lack of native support for distributed transactions in most setups.
**Approach:**
- **Two-Phase Commit:** Implement a two-phase commit protocol, which can be cumbersome.
- **Eventual Consistency:** Design the system to handle eventual consistency, ensuring that data across databases synchronize over time.
**Recommendation:**
For simplicity and given the project scope, adopt **eventual consistency** and ensure robust error handling and retry mechanisms.
---
#### 15. **Monitor and Optimize Database Performance**
Regularly monitor database performance and optimize queries to ensure responsiveness.
##### a. **Use Sequelize Logging**
Enable Sequelize's logging to monitor executed queries.
**Update `config/config.js`:**
```javascript
// config/config.js
development: {
// ... existing configurations
logging: (msg) => console.log(msg), // Enable logging
},
// ... other environments
```
**Disable Logging in Production:**
```javascript
production: {
// ... existing configurations
logging: false, // Disable logging to enhance performance
},
```
##### b. **Analyze Slow Queries**
Identify and optimize slow-running queries.
**Tools:**
- **PostgreSQL:** Use the `pg_stat_statements` extension to track query performance.
**Enable `pg_stat_statements`:**
```sql
-- In psql
CREATE EXTENSION pg_stat_statements;
```
**Querying Slow Queries:**
```sql
SELECT
query,
calls,
total_time,
mean_time,
min_time,
max_time
FROM
pg_stat_statements
ORDER BY
total_time DESC
LIMIT 10;
```
- **MongoDB:** Use the `explain` method to analyze query performance.
**Example:**
```javascript
// In Mongoose
LearningMaterial.find({ courseId: 'course-uuid' }).explain('executionStats');
```
##### c. **Implement Indexing Strategically**
Avoid over-indexing, which can degrade write performance. Index fields that are frequently queried or used in join operations.
**Example: Indexing `courseId` in Learning Materials**
```javascript
// models/mongoose/LearningMaterial.js
const LearningMaterialSchema = new mongoose.Schema(
{
// ... existing fields
courseId: {
type: String,
required: true,
index: true // Creates an index
},
// ... other fields
},
{ timestamps: true }
);
```
---
#### 16. **Ensure Data Backup and Recovery**
Implement reliable backup and recovery strategies to prevent data loss.
##### a. **Automate Backups**
Set up automated backup schedules for both PostgreSQL and MongoDB.
**PostgreSQL:**
- **Cron Job Example:**
```bash
# Backup daily at 2 AM
0 2 * * * pg_dump -U your_pg_username -h your_pg_host -F c -b -v -f /backup/educonnect_dev_$(date +\%F).backup educonnect_dev
```
**MongoDB:**
- **Cron Job Example:**
```bash
# Backup daily at 3 AM
0 3 * * * mongodump --uri="mongodb://username:password@localhost:27017/educonnect" --out=/backup/educonnect_$(date +\%F)/
```
##### b. **Test Recovery Procedures**
Regularly test backups to ensure they can be restored successfully.
**PostgreSQL Restore Example:**
```bash
pg_restore -U your_pg_username -h your_pg_host -d educonnect_dev_restored -v /backup/educonnect_dev.backup
```
**MongoDB Restore Example:**
```bash
mongorestore --uri="mongodb://username:password@localhost:27017/educonnect_restored" /backup/educonnect/
```
---
#### 17. **Implement Data Seeding for Development and Testing**
Populate databases with initial data to facilitate development and testing.
##### a. **PostgreSQL Seeders**
Already covered in **Section 7.b**, but ensure comprehensive seed data.
**Example: Seeding Multiple Users and Courses**
```javascript
// seeders/XXXXXX-multiple-users-and-courses.js
'use strict';
const bcrypt = require('bcrypt');
module.exports = {
up: async (queryInterface, Sequelize) => {
const hashedPassword = await bcrypt.hash('password123', 10);
await queryInterface.bulkInsert('Users', [
{
id: 'user-uuid-1',
username: 'john_doe',
email: 'john@example.com',
password: hashedPassword,
roles: ['student'],
createdAt: new Date(),
updatedAt: new Date()
},
{
id: 'user-uuid-2',
username: 'jane_instructor',
email: 'jane@example.com',
password: hashedPassword,
roles: ['instructor'],
createdAt: new Date(),
updatedAt: new Date()
}
]);
await queryInterface.bulkInsert('Courses', [
{
id: 'course-uuid-1',
title: 'Introduction to Node.js',
description: 'Learn the basics of Node.js.',
category: 'Programming',
instructorId: 'user-uuid-2',
createdAt: new Date(),
updatedAt: new Date()
}
]);
// Add more seed data as needed
},
down: async (queryInterface, Sequelize) => {
await queryInterface.bulkDelete('Courses', null, {});
await queryInterface.bulkDelete('Users', null, {});
}
};
```
##### b. **MongoDB Seeders**
Create scripts to seed MongoDB collections.
**Example: Seeding Learning Materials**
```javascript
// scripts/seedLearningMaterials.js
require('dotenv').config();
const mongoose = require('mongoose');
const LearningMaterial = require('../src/models/mongoose/LearningMaterial');
const logger = require('../src/utils/logger');
const seedLearningMaterials = async () => {
try {
await mongoose.connect(process.env.MONGO_URI, {
useNewUrlParser: true,
useUnifiedTopology: true
});
const materials = [
{
title: 'Node.js Basics',
content: 'Introduction to Node.js',
courseId: 'course-uuid-1',
resourceType: 'video',
url: 'https://example.com/video1.mp4'
},
{
title: 'Advanced Node.js',
content: 'Deep dive into Node.js',
courseId: 'course-uuid-1',
resourceType: 'document',
url: 'https://example.com/doc1.pdf'
}
];
await LearningMaterial.insertMany(materials);
logger.info('Learning materials seeded successfully');
process.exit(0);
} catch (error) {
logger.error('Error seeding learning materials:', error);
process.exit(1);
}
};
seedLearningMaterials();
```
**Run the Seeder:**
```bash
node scripts/seedLearningMaterials.js
```
---
#### 18. **Ensure Data Consistency Across Databases**
While PostgreSQL and MongoDB operate independently, maintaining consistency between related data is crucial.
##### a. **Referential Integrity**
Since MongoDB does not enforce referential integrity, ensure that references (like `userId` and `courseId`) are validated at the application level.
**Example: Validating `courseId` Before Adding Learning Material**
```javascript
// src/controllers/learningMaterialController.js
const courseService = require('../services/courseService');
const learningMaterialService = require('../services/learningMaterialService');
const CustomError = require('../utils/CustomError');
exports.addLearningMaterial = async (req, res, next) => {
try {
const { courseId, title, content, resourceType, url } = req.body;
// Validate that the course exists in PostgreSQL
const course = await courseService.findCourseById(courseId);
if (!course) {
throw new CustomError('Course not found', 404);
}
// Create Learning Material in MongoDB
const material = await learningMaterialService.createLearningMaterial({
courseId,
title,
content,
resourceType,
url
});
res.status(201).json({ success: true, data: material });
} catch (error) {
next(error);
}
};
```
##### b. **Cascade Operations**
Implement cascade operations manually where necessary.
**Example: Deleting a Course and Its Learning Materials**
```javascript
// src/services/courseService.js
const { Course, Quiz } = require('../models');
const LearningMaterial = require('../models/mongoose/LearningMaterial');
const deleteCourse = async (courseId) => {
const transaction = await sequelize.transaction();
try {
// Delete quizzes associated with the course
await Quiz.destroy({ where: { courseId }, transaction });
// Delete the course
await Course.destroy({ where: { id: courseId }, transaction });
// Commit PostgreSQL transactions
await transaction.commit();
// Delete learning materials from MongoDB
await LearningMaterial.deleteMany({ courseId });
return true;
} catch (error) {
await transaction.rollback();
throw new CustomError('Failed to delete course', 500);
}
};
module.exports = {
deleteCourse,
// ... other exports
};
```
**Notes:**
- **Consistency Checks:** Regularly verify that references between databases remain consistent.
- **Error Handling:** Implement robust error handling to manage failures in cross-database operations.
---
#### 19. **Implement Pagination and Query Optimization**
Efficient data retrieval is essential, especially for endpoints that return large datasets.
##### a. **Sequelize Pagination**
**Example: Paginating Courses**
```javascript
// src/services/courseService.js
const getAllCourses = async (page = 1, limit = 10) => {
const offset = (page - 1) * limit;
const { count, rows } = await Course.findAndCountAll({
limit,
offset,
order: [['createdAt', 'DESC']]
});
return {
totalItems: count,
totalPages: Math.ceil(count / limit),
currentPage: page,
courses: rows
};
};
module.exports = {
getAllCourses,
// ... other exports
};
```
##### b. **Mongoose Pagination**
**Example: Paginating Learning Materials**
```javascript
// src/services/learningMaterialService.js
const getMaterialsByCourseId = async (courseId, page = 1, limit = 10) => {
const skip = (page - 1) * limit;
const materials = await LearningMaterial.find({ courseId })
.skip(skip)
.limit(limit)
.sort({ createdAt: -1 });
const total = await LearningMaterial.countDocuments({ courseId });
return {
totalItems: total,
totalPages: Math.ceil(total / limit),
currentPage: page,
materials
};
};
module.exports = {
getMaterialsByCourseId,
// ... other exports
};
```
##### c. **Optimize Queries**
- **Select Specific Fields:** Retrieve only necessary fields to reduce data transfer.
**Sequelize Example:**
```javascript
const user = await User.findByPk(id, {
attributes: ['id', 'username', 'email']
});
```
- **Use Aggregations in MongoDB:** For complex data retrieval, utilize MongoDB's aggregation framework.
**Example: Aggregating User Progress**
```javascript
const aggregateProgress = async (userId) => {
return await UserProgress.aggregate([
{ $match: { userId } },
{ $group: { _id: '$courseId', totalCompleted: { $sum: { $size: '$completedModules' } } } }
]);
};
```
---
#### 20. **Implement Soft Deletes (Optional)**
Soft deletes allow you to mark records as deleted without permanently removing them, facilitating data recovery and auditing.
##### a. **Sequelize Soft Deletes**
**Add `deletedAt` Field and Enable Paranoid Mode**
Modify the model definitions to include `deletedAt` and enable paranoid mode.
**Example: User Model with Soft Deletes**
```javascript
// models/user.js
// ... existing code
User.init(
{
// ... existing fields
},
{
sequelize,
modelName: 'User',
tableName: 'Users',
timestamps: true,
paranoid: true, // Enables soft deletes
deletedAt: 'deletedAt'
}
);
```
**Migration to Add `deletedAt` Column:**
```bash
npx sequelize-cli migration:generate --name add-deletedAt-to-users
```
**`migrations/XXXXXX-add-deletedAt-to-users.js`:**
```javascript
'use strict';
module.exports = {
up: async (queryInterface, Sequelize) => {
await queryInterface.addColumn('Users', 'deletedAt', {
type: Sequelize.DATE,
allowNull: true
});
},
down: async (queryInterface, Sequelize) => {
await queryInterface.removeColumn('Users', 'deletedAt');
}
};
```
**Run Migration:**
```bash
npx sequelize-cli db:migrate
```
**Notes:**
- **Paranoid Mode:** When enabled, Sequelize excludes soft-deleted records from query results by default.
- **Restoring Records:** Implement methods to restore soft-deleted records if necessary.
---
### **Summary**
In **Step 3: Database Integration**, we've successfully:
- **Chosen and Configured Databases:**
- **PostgreSQL** with **Sequelize** for relational data.
- **MongoDB** with **Mongoose** for unstructured data.
- **Installed Necessary Dependencies:** Included Sequelize, Sequelize CLI, pg, pg-hstore for PostgreSQL, and Mongoose for MongoDB.
- **Configured Database Connections:** Set up configuration files and environment variables for secure and flexible connections.
- **Defined Models and Schemas:**
- **Sequelize Models:** User, Course, Quiz with associations.
- **Mongoose Schemas:** LearningMaterial, UserProgress.
- **Implemented Repository and Service Layers:** Encapsulated database interactions for better maintainability and testability.
- **Handled Migrations and Seeders:** Managed database schema changes and populated initial data.
- **Ensured Data Validation and Security:** Applied validations at the model and request levels, secured database credentials, and implemented connection pooling.
- **Optimized Performance:** Configured caching with Redis, implemented pagination, and optimized queries.
- **Established Backup and Recovery Procedures:** Automated backups and tested recovery processes to safeguard data.
- **Ensured Data Consistency:** Maintained references between PostgreSQL and MongoDB and handled cross-database operations carefully.
- **Optional Enhancements:** Implemented soft deletes for data recovery and auditing purposes.
---
### **Next Steps**
With the databases integrated and configured, the next step is to **Step 4: Implement Core Functionality**. In this phase, we'll develop the main features and API endpoints, ensuring alignment with your **project_requirements** such as user authentication, course management, progress tracking, adaptive learning algorithms, video streaming integration, and the quiz system.
Feel free to ask any questions or request further assistance regarding **Step 3: Database Integration**!
===
### Step 4: Implement Core Functionality
Great! Moving forward, in **Step 4: Implement Core Functionality**, we'll develop the main features and API endpoints of the **EduConnect: Adaptive Learning Backend**. This step ensures that the backend aligns with your **project_requirements**, including user authentication, course management, progress tracking, adaptive learning algorithms, video streaming integration, and the quiz system.
---
#### 1. **Set Up Express Middleware and Routing Structure**
Before diving into specific functionalities, ensure that your Express application is set up to handle routing and middleware effectively.
**a. Update `src/app.js`**
Create an `app.js` file to configure middleware and routes, separating it from `index.js` for better organization.
```javascript
// src/app.js
const express = require('express');
const cors = require('cors');
const helmet = require('helmet'); // For security headers
const rateLimiter = require('./middleware/rateLimiter');
const errorHandler = require('./middleware/errorHandler');
const swaggerUi = require('swagger-ui-express');
const swaggerJsdoc = require('swagger-jsdoc');
const { graphqlHTTP } = require('express-graphql');
const schema = require('./graphql/schema');
const resolvers = require('./graphql/resolvers');
const app = express();
// Middleware
app.use(helmet());
app.use(express.json());
app.use(express.urlencoded({ extended: false }));
app.use(cors());
// Rate Limiting
app.use('/v1', rateLimiter);
// Routes
const userRoutes = require('./routes/v1/userRoutes');
const courseRoutes = require('./routes/v1/courseRoutes');
const progressRoutes = require('./routes/v1/progressRoutes');
const quizRoutes = require('./routes/v1/quizRoutes');
const learningMaterialRoutes = require('./routes/v1/learningMaterialRoutes');
app.use('/v1/users', userRoutes);
app.use('/v1/courses', courseRoutes);
app.use('/v1/progress', progressRoutes);
app.use('/v1/quizzes', quizRoutes);
app.use('/v1/learning-materials', learningMaterialRoutes);
// GraphQL Endpoint
app.use(
'/graphql',
graphqlHTTP({
schema,
rootValue: resolvers,
graphiql: process.env.NODE_ENV !== 'production'
})
);
// API Documentation
const swaggerOptions = {
definition: {
openapi: '3.0.0',
info: {
title: 'EduConnect API',
version: '1.0.0',
description: 'API documentation for EduConnect Backend',
},
servers: [
{
url: 'https://api.educonnect.com/v1',
},
],
},
apis: ['./src/routes/v1/*.js'], // Path to the API docs
};
const swaggerSpec = swaggerJsdoc(swaggerOptions);
app.use('/api-docs', swaggerUi.serve, swaggerUi.setup(swaggerSpec));
// Error Handling Middleware
app.use(errorHandler);
module.exports = app;
```
**b. Update `src/index.js`**
Modify `index.js` to use the newly created `app.js`.
```javascript
// src/index.js
require('dotenv').config();
const http = require('http');
const app = require('./app');
const { sequelize } = require('./models'); // Sequelize instance
const connectDB = require('../config/mongoose');
const { connectRedis } = require('../config/redis');
const logger = require('./utils/logger');
const PORT = process.env.PORT || 3000;
const httpServer = http.createServer(app);
// Optional: Initialize WebSocket server here if using libraries like Socket.io
const startServer = async () => {
try {
// Connect to PostgreSQL
await sequelize.authenticate();
logger.info('PostgreSQL connected successfully.');
// Connect to MongoDB
await connectDB();
// Connect to Redis
await connectRedis();
// Start the server
httpServer.listen(PORT, () => {
logger.info(`Server is running on port ${PORT}`);
});
} catch (error) {
logger.error('Database connection failed:', error);
process.exit(1);
}
};
startServer();
```
---
#### 2. **Develop User Authentication and Management**
Implement user registration, login, profile management, and role-based access control.
**a. **User Routes**
**`src/routes/v1/userRoutes.js`**
```javascript
// src/routes/v1/userRoutes.js
const express = require('express');
const router = express.Router();
const { register, login, getProfile, updateProfile, deleteUser, refreshToken } = require('../../controllers/userController');
const { body } = require('express-validator');
const validate = require('../../middleware/validate');
const authMiddleware = require('../../middleware/authMiddleware');
/**
* @swagger
* tags:
* name: Users
* description: User management and authentication
*/
// Register User
router.post(
'/register',
[
body('username').notEmpty().withMessage('Username is required'),
body('email').isEmail().withMessage('Valid email is required'),
body('password').isLength({ min: 6 }).withMessage('Password must be at least 6 characters long'),
],
validate,
register
);
// Login User
router.post(
'/login',
[
body('email').isEmail().withMessage('Valid email is required'),
body('password').notEmpty().withMessage('Password is required'),
],
validate,
login
);
// Refresh Token
router.post('/refresh-token', refreshToken);
// Get User Profile
router.get('/profile', authMiddleware, getProfile);
// Update User Profile
router.put(
'/profile',
authMiddleware,
[
body('username').optional().notEmpty().withMessage('Username cannot be empty'),
body('email').optional().isEmail().withMessage('Valid email is required'),
// Add more validations as needed
],
validate,
updateProfile
);
// Delete User
router.delete('/profile', authMiddleware, deleteUser);
module.exports = router;
```
**b. **User Controller**
**`src/controllers/userController.js`**
```javascript
// src/controllers/userController.js
const userService = require('../services/userService');
const jwt = require('jsonwebtoken');
const bcrypt = require('bcrypt');
const CustomError = require('../utils/CustomError');
// Helper to generate tokens
const generateToken = (user) => {
return jwt.sign(
{ id: user.id, roles: user.roles },
process.env.JWT_SECRET,
{ expiresIn: '1h' }
);
};
const generateRefreshToken = (user) => {
return jwt.sign(
{ id: user.id },
process.env.JWT_REFRESH_SECRET,
{ expiresIn: '7d' }
);
};
// Register User
exports.register = async (req, res, next) => {
try {
const { username, email, password } = req.body;
// Check if user exists
const existingUser = await userService.findUserByEmail(email);
if (existingUser) {
throw new CustomError('User already exists with this email', 400);
}
// Hash password
const hashedPassword = await bcrypt.hash(password, 10);
// Create user
const user = await userService.createUser({
username,
email,
password: hashedPassword,
roles: ['student'],
});
// Generate tokens
const token = generateToken(user);
const refreshToken = generateRefreshToken(user);
// Optionally, store refresh token in Redis or database
res.status(201).json({
success: true,
data: {
id: user.id,
username: user.username,
email: user.email,
token,
refreshToken,
},
});
} catch (error) {
next(error);
}
};
// Login User
exports.login = async (req, res, next) => {
try {
const { email, password } = req.body;
// Find user
const user = await userService.findUserByEmail(email);
if (!user) {
throw new CustomError('Invalid email or password', 401);
}
// Compare password
const isMatch = await bcrypt.compare(password, user.password);
if (!isMatch) {
throw new CustomError('Invalid email or password', 401);
}
// Generate tokens
const token = generateToken(user);
const refreshToken = generateRefreshToken(user);
// Optionally, store refresh token in Redis or database
res.status(200).json({
success: true,
data: {
token,
refreshToken,
},
});
} catch (error) {
next(error);
}
};
// Refresh Token
exports.refreshToken = async (req, res, next) => {
try {
const { refreshToken } = req.body;
if (!refreshToken) {
throw new CustomError('Refresh token is required', 400);
}
// Verify refresh token
const decoded = jwt.verify(refreshToken, process.env.JWT_REFRESH_SECRET);
const user = await userService.findUserById(decoded.id);
if (!user) {
throw new CustomError('User not found', 404);
}
// Generate new access token
const newToken = generateToken(user);
res.status(200).json({
success: true,
data: {
token: newToken,
},
});
} catch (error) {
if (error.name === 'TokenExpiredError') {
return next(new CustomError('Refresh token expired', 401));
}
next(error);
}
};
// Get User Profile
exports.getProfile = async (req, res, next) => {
try {
const user = await userService.findUserById(req.user.id);
if (!user) {
throw new CustomError('User not found', 404);
}
res.status(200).json({
success: true,
data: {
id: user.id,
username: user.username,
email: user.email,
roles: user.roles,
},
});
} catch (error) {
next(error);
}
};
// Update User Profile
exports.updateProfile = async (req, res, next) => {
try {
const updates = req.body;
const user = await userService.updateUser(req.user.id, updates);
if (!user) {
throw new CustomError('User not found', 404);
}
res.status(200).json({
success: true,
data: {
id: user.id,
username: user.username,
email: user.email,
roles: user.roles,
},
});
} catch (error) {
next(error);
}
};
// Delete User
exports.deleteUser = async (req, res, next) => {
try {
const success = await userService.deleteUser(req.user.id);
if (!success) {
throw new CustomError('User not found', 404);
}
res.status(200).json({
success: true,
message: 'User account deleted successfully',
});
} catch (error) {
next(error);
}
};
```
**c. **User Service**
**`src/services/userService.js`**
```javascript
// src/services/userService.js
const userRepository = require('../repositories/userRepository');
const createUser = async (userData) => {
return await userRepository.create(userData);
};
const findUserByEmail = async (email) => {
return await userRepository.findByEmail(email);
};
const findUserById = async (id) => {
return await userRepository.findById(id);
};
const updateUser = async (id, updates) => {
return await userRepository.update(id, updates);
};
const deleteUser = async (id) => {
return await userRepository.delete(id);
};
module.exports = {
createUser,
findUserByEmail,
findUserById,
updateUser,
deleteUser,
};
```
**d. **User Repository**
**`src/repositories/userRepository.js`**
```javascript
// src/repositories/userRepository.js
const { User, Course } = require('../models');
class UserRepository {
async create(userData) {
return await User.create(userData);
}
async findByEmail(email) {
return await User.findOne({ where: { email } });
}
async findById(id) {
return await User.findByPk(id, {
attributes: ['id', 'username', 'email', 'roles', 'createdAt', 'updatedAt'],
include: [{ model: Course, as: 'courses', attributes: ['id', 'title'] }],
});
}
async update(id, updates) {
const user = await User.findByPk(id);
if (!user) return null;
await user.update(updates);
return user;
}
async delete(id) {
const user = await User.findByPk(id);
if (!user) return false;
await user.destroy(); // Soft delete if paranoid is enabled
return true;
}
}
module.exports = new UserRepository();
```
**e. **Authentication Middleware**
**`src/middleware/authMiddleware.js`**
```javascript
// src/middleware/authMiddleware.js
const jwt = require('jsonwebtoken');
const CustomError = require('../utils/CustomError');
const authMiddleware = (req, res, next) => {
const authHeader = req.headers.authorization;
if (!authHeader || !authHeader.startsWith('Bearer ')) {
return next(new CustomError('Authorization header missing or malformed', 401));
}
const token = authHeader.split(' ')[1];
try {
const decoded = jwt.verify(token, process.env.JWT_SECRET);
req.user = {
id: decoded.id,
roles: decoded.roles,
};
next();
} catch (error) {
if (error.name === 'TokenExpiredError') {
return next(new CustomError('Token expired', 401));
}
next(new CustomError('Invalid token', 401));
}
};
module.exports = authMiddleware;
```
---
#### 3. **Implement Course Management**
Enable instructors/admins to create, read, update, and delete courses.
**a. **Course Routes**
**`src/routes/v1/courseRoutes.js`**
```javascript
// src/routes/v1/courseRoutes.js
const express = require('express');
const router = express.Router();
const { createCourse, getAllCourses, getCourseById, updateCourse, deleteCourse } = require('../../controllers/courseController');
const { body, param } = require('express-validator');
const validate = require('../../middleware/validate');
const authMiddleware = require('../../middleware/authMiddleware');
const roleMiddleware = require('../../middleware/roleMiddleware');
/**
* @swagger
* tags:
* name: Courses
* description: Course management
*/
// Create Course
router.post(
'/',
authMiddleware,
roleMiddleware(['admin', 'instructor']),
[
body('title').notEmpty().withMessage('Title is required'),
body('description').notEmpty().withMessage('Description is required'),
body('category').notEmpty().withMessage('Category is required'),
body('videos').isArray().withMessage('Videos should be an array of URLs'),
body('materials').isArray().withMessage('Materials should be an array of URLs'),
],
validate,
createCourse
);
// Get All Courses with Pagination and Filtering
router.get('/', getAllCourses);
// Get Course by ID
router.get(
'/:courseId',
[
param('courseId').isUUID().withMessage('Valid courseId is required'),
],
validate,
getCourseById
);
// Update Course
router.put(
'/:courseId',
authMiddleware,
roleMiddleware(['admin', 'instructor']),
[
param('courseId').isUUID().withMessage('Valid courseId is required'),
body('title').optional().notEmpty().withMessage('Title cannot be empty'),
body('description').optional().notEmpty().withMessage('Description cannot be empty'),
body('category').optional().notEmpty().withMessage('Category cannot be empty'),
body('videos').optional().isArray().withMessage('Videos should be an array of URLs'),
body('materials').optional().isArray().withMessage('Materials should be an array of URLs'),
],
validate,
updateCourse
);
// Delete Course
router.delete(
'/:courseId',
authMiddleware,
roleMiddleware(['admin', 'instructor']),
[
param('courseId').isUUID().withMessage('Valid courseId is required'),
],
validate,
deleteCourse
);
module.exports = router;
```
**b. **Role-Based Access Control Middleware**
Ensure only users with specific roles can access certain endpoints.
**`src/middleware/roleMiddleware.js`**
```javascript
// src/middleware/roleMiddleware.js
const CustomError = require('../utils/CustomError');
const roleMiddleware = (allowedRoles) => {
return (req, res, next) => {
const userRoles = req.user.roles;
const hasRole = userRoles.some(role => allowedRoles.includes(role));
if (!hasRole) {
return next(new CustomError('Access denied: insufficient permissions', 403));
}
next();
};
};
module.exports = roleMiddleware;
```
**c. **Course Controller**
**`src/controllers/courseController.js`**
```javascript
// src/controllers/courseController.js
const courseService = require('../services/courseService');
const CustomError = require('../utils/CustomError');
// Create Course
exports.createCourse = async (req, res, next) => {
try {
const { title, description, category, videos, materials } = req.body;
const instructorId = req.user.id;
const course = await courseService.createCourse({
title,
description,
category,
videos,
materials,
instructorId,
});
res.status(201).json({
success: true,
data: course,
});
} catch (error) {
next(error);
}
};
// Get All Courses with Pagination and Filtering
exports.getAllCourses = async (req, res, next) => {
try {
const { page = 1, limit = 10, category, search } = req.query;
const courses = await courseService.getAllCourses({ page, limit, category, search });
res.status(200).json({
success: true,
data: courses,
});
} catch (error) {
next(error);
}
};
// Get Course by ID
exports.getCourseById = async (req, res, next) => {
try {
const { courseId } = req.params;
const course = await courseService.getCourseById(courseId);
if (!course) {
throw new CustomError('Course not found', 404);
}
res.status(200).json({
success: true,
data: course,
});
} catch (error) {
next(error);
}
};
// Update Course
exports.updateCourse = async (req, res, next) => {
try {
const { courseId } = req.params;
const updates = req.body;
// Verify that the user is the instructor or admin
const course = await courseService.getCourseById(courseId);
if (!course) {
throw new CustomError('Course not found', 404);
}
if (req.user.roles.includes('admin') || course.instructorId === req.user.id) {
const updatedCourse = await courseService.updateCourse(courseId, updates);
res.status(200).json({
success: true,
data: updatedCourse,
});
} else {
throw new CustomError('Access denied: not the course instructor', 403);
}
} catch (error) {
next(error);
}
};
// Delete Course
exports.deleteCourse = async (req, res, next) => {
try {
const { courseId } = req.params;
// Verify that the user is the instructor or admin
const course = await courseService.getCourseById(courseId);
if (!course) {
throw new CustomError('Course not found', 404);
}
if (req.user.roles.includes('admin') || course.instructorId === req.user.id) {
await courseService.deleteCourse(courseId);
res.status(200).json({
success: true,
message: 'Course deleted successfully',
});
} else {
throw new CustomError('Access denied: not the course instructor', 403);
}
} catch (error) {
next(error);
}
};
```
**d. **Course Service**
**`src/services/courseService.js`**
```javascript
// src/services/courseService.js
const courseRepository = require('../repositories/courseRepository');
const learningMaterialService = require('./learningMaterialService');
const quizService = require('./quizService');
const createCourse = async (courseData) => {
// Assuming courseData contains title, description, category, videos, materials, instructorId
const course = await courseRepository.create(courseData);
// Optionally, create learning materials and quizzes here or handle separately
// Example:
// await learningMaterialService.createLearningMaterials(course.id, courseData.materials);
return course;
};
const getAllCourses = async ({ page, limit, category, search }) => {
return await courseRepository.findAll({ page, limit, category, search });
};
const getCourseById = async (courseId) => {
return await courseRepository.findById(courseId);
};
const updateCourse = async (courseId, updates) => {
return await courseRepository.update(courseId, updates);
};
const deleteCourse = async (courseId) => {
return await courseRepository.delete(courseId);
};
module.exports = {
createCourse,
getAllCourses,
getCourseById,
updateCourse,
deleteCourse,
};
```
**e. **Course Repository**
**`src/repositories/courseRepository.js`**
```javascript
// src/repositories/courseRepository.js
const { Course, Quiz, User } = require('../models');
const { Op } = require('sequelize');
class CourseRepository {
async create(courseData) {
return await Course.create(courseData);
}
async findAll({ page, limit, category, search }) {
const offset = (page - 1) * limit;
const whereClause = {};
if (category) {
whereClause.category = category;
}
if (search) {
whereClause[Op.or] = [
{ title: { [Op.iLike]: `%${search}%` } },
{ description: { [Op.iLike]: `%${search}%` } },
];
}
const { count, rows } = await Course.findAndCountAll({
where: whereClause,
limit: parseInt(limit),
offset: parseInt(offset),
order: [['createdAt', 'DESC']],
include: [{ model: User, as: 'instructor', attributes: ['id', 'username', 'email'] }],
});
return {
totalItems: count,
totalPages: Math.ceil(count / limit),
currentPage: parseInt(page),
courses: rows,
};
}
async findById(courseId) {
return await Course.findByPk(courseId, {
include: [
{ model: User, as: 'instructor', attributes: ['id', 'username', 'email'] },
{ model: Quiz, as: 'quizzes' },
],
});
}
async update(courseId, updates) {
const course = await Course.findByPk(courseId);
if (!course) return null;
await course.update(updates);
return course;
}
async delete(courseId) {
// Implement cascade deletion if needed, or handle in service
return await Course.destroy({ where: { id: courseId } });
}
}
module.exports = new CourseRepository();
```
---
#### 4. **Implement Progress Tracking**
Allow users to track their progress through courses, including completed modules and quiz scores.
**a. **Progress Routes**
**`src/routes/v1/progressRoutes.js`**
```javascript
// src/routes/v1/progressRoutes.js
const express = require('express');
const router = express.Router();
const { getUserProgress, updateUserProgress } = require('../../controllers/progressController');
const { body } = require('express-validator');
const validate = require('../../middleware/validate');
const authMiddleware = require('../../middleware/authMiddleware');
/**
* @swagger
* tags:
* name: Progress
* description: User progress tracking
*/
// Get User Progress
router.get('/', authMiddleware, getUserProgress);
// Update User Progress
router.put(
'/',
authMiddleware,
[
body('courseId').isUUID().withMessage('Valid courseId is required'),
body('completedModules').isArray().withMessage('completedModules should be an array of module identifiers'),
body('quizScores').isObject().withMessage('quizScores should be an object with quizId as keys and scores as values'),
],
validate,
updateUserProgress
);
module.exports = router;
```
**b. **Progress Controller**
**`src/controllers/progressController.js`**
```javascript
// src/controllers/progressController.js
const progressService = require('../services/progressService');
const CustomError = require('../utils/CustomError');
exports.getUserProgress = async (req, res, next) => {
try {
const userId = req.user.id;
const progress = await progressService.getUserProgress(userId);
res.status(200).json({
success: true,
data: progress,
});
} catch (error) {
next(error);
}
};
exports.updateUserProgress = async (req, res, next) => {
try {
const userId = req.user.id;
const { courseId, completedModules, quizScores } = req.body;
const updatedProgress = await progressService.updateUserProgress(userId, { courseId, completedModules, quizScores });
res.status(200).json({
success: true,
data: updatedProgress,
});
} catch (error) {
next(error);
}
};
```
**c. **Progress Service**
**`src/services/progressService.js`**
```javascript
// src/services/progressService.js
const userProgressRepository = require('../repositories/userProgressRepository');
const courseService = require('./courseService');
const CustomError = require('../utils/CustomError');
const getUserProgress = async (userId) => {
return await userProgressRepository.findByUserId(userId);
};
const updateUserProgress = async (userId, { courseId, completedModules, quizScores }) => {
// Validate course existence
const course = await courseService.getCourseById(courseId);
if (!course) {
throw new CustomError('Course not found', 404);
}
// Update progress
const progress = await userProgressRepository.updateProgress(userId, courseId, { completedModules, quizScores });
return progress;
};
module.exports = {
getUserProgress,
updateUserProgress,
};
```
**d. **Progress Repository**
**`src/repositories/userProgressRepository.js`**
```javascript
// src/repositories/userProgressRepository.js
const UserProgress = require('../models/mongoose/UserProgress');
class UserProgressRepository {
async findByUserId(userId) {
return await UserProgress.find({ userId }).lean();
}
async updateProgress(userId, courseId, updates) {
const progress = await UserProgress.findOne({ userId, courseId });
if (progress) {
// Update existing progress
if (updates.completedModules) {
progress.completedModules = [...new Set([...progress.completedModules, ...updates.completedModules])];
}
if (updates.quizScores) {
Object.keys(updates.quizScores).forEach(quizId => {
progress.quizScores.set(quizId, updates.quizScores[quizId]);
});
}
if (updates.adaptiveData) {
progress.adaptiveData = { ...progress.adaptiveData, ...updates.adaptiveData };
}
await progress.save();
return progress;
} else {
// Create new progress
const newProgress = new UserProgress({
userId,
courseId,
completedModules: updates.completedModules || [],
quizScores: updates.quizScores || {},
adaptiveData: updates.adaptiveData || {},
});
await newProgress.save();
return newProgress;
}
}
}
module.exports = new UserProgressRepository();
```
---
#### 5. **Integrate Adaptive Learning Algorithms**
Implement adaptive learning algorithms that adjust course content based on user progress and performance.
**a. **Adaptive Learning Service**
**`src/services/adaptiveLearningService.js`**
```javascript
// src/services/adaptiveLearningService.js
const userProgressService = require('./progressService');
const learningMaterialService = require('./learningMaterialService');
const quizService = require('./quizService');
const adjustLearningPath = async (userId, courseId) => {
// Fetch user progress
const progress = await userProgressService.getUserProgress(userId);
const courseProgress = progress.find(p => p.courseId === courseId);
if (!courseProgress) {
// Initialize progress if not exists
await userProgressService.updateUserProgress(userId, { courseId, completedModules: [], quizScores: {} });
return;
}
// Analyze quiz scores to determine areas of improvement
const lowScores = Object.entries(courseProgress.quizScores)
.filter(([quizId, score]) => score < 70)
.map(([quizId]) => quizId);
// Recommend additional materials or modules based on low scores
if (lowScores.length > 0) {
const recommendedMaterials = await learningMaterialService.getRecommendedMaterials(courseId, lowScores);
// Update adaptive data or notify user
// This can be expanded based on specific requirements
return recommendedMaterials;
}
// Further adaptive logic can be implemented here
};
module.exports = {
adjustLearningPath,
};
```
**b. **Invoke Adaptive Learning After Quiz Submission**
Modify the quiz submission controller to trigger the adaptive learning adjustment.
**`src/controllers/quizController.js`**
```javascript
// src/controllers/quizController.js
const quizService = require('../services/quizService');
const adaptiveLearningService = require('../services/adaptiveLearningService');
const CustomError = require('../utils/CustomError');
exports.submitQuiz = async (req, res, next) => {
try {
const { quizId } = req.params;
const { answers } = req.body;
const userId = req.user.id;
// Submit quiz and get results
const quizResult = await quizService.submitQuiz(userId, quizId, answers);
// Adjust learning path based on results
await adaptiveLearningService.adjustLearningPath(userId, quizResult.courseId);
res.status(200).json({
success: true,
data: quizResult,
});
} catch (error) {
next(error);
}
};
```
---
#### 6. **Integrate Video Streaming**
Allow users to stream video content within courses.
**a. **Video Streaming Service**
Depending on your video hosting strategy, you can either:
- **Host Videos on a CDN:** Use services like AWS S3 with CloudFront, Vimeo, or other video platforms.
- **Stream Directly from Server:** Not recommended for scalability but possible for small projects.
**Example: Serving Videos from AWS S3 with CloudFront**
1. **Upload Videos to S3:**
- Organize videos into buckets or folders based on courses.
2. **Set Up CloudFront Distribution:**
- Configure CloudFront to serve videos from S3.
- Enable streaming features as needed.
3. **Store Video URLs in Course Data:**
- In your course model, store the CloudFront URLs for video content.
**b. **Learning Material Service Update**
Ensure that video URLs are correctly integrated into learning materials.
**`src/services/learningMaterialService.js`**
```javascript
// src/services/learningMaterialService.js
const LearningMaterial = require('../models/mongoose/LearningMaterial');
const createLearningMaterial = async (materialData) => {
const material = new LearningMaterial(materialData);
return await material.save();
};
const getMaterialsByCourseId = async (courseId) => {
return await LearningMaterial.find({ courseId });
};
const getRecommendedMaterials = async (courseId, lowScores) => {
// Implement logic to fetch recommended materials based on lowScores
// For example, fetch materials related to specific modules or topics
// This is a placeholder implementation
return await LearningMaterial.find({ courseId, resourceType: 'document' });
};
module.exports = {
createLearningMaterial,
getMaterialsByCourseId,
getRecommendedMaterials,
};
```
**c. **Serve Video Content Securely**
Ensure that video URLs are secured, possibly with signed URLs that expire after a certain time to prevent unauthorized access.
**Example: Generating Signed URLs with AWS SDK**
```javascript
// src/services/videoService.js
const AWS = require('aws-sdk');
const CustomError = require('../utils/CustomError');
const s3 = new AWS.S3({
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
region: process.env.AWS_REGION,
});
const generateSignedUrl = (bucketName, key, expires = 3600) => {
const params = {
Bucket: bucketName,
Key: key,
Expires: expires, // URL expiration time in seconds
};
return s3.getSignedUrl('getObject', params);
};
const getVideoUrl = async (courseId, videoKey) => {
try {
const signedUrl = generateSignedUrl(process.env.AWS_S3_BUCKET, `courses/${courseId}/videos/${videoKey}`);
return signedUrl;
} catch (error) {
throw new CustomError('Failed to generate video URL', 500);
}
};
module.exports = {
getVideoUrl,
};
```
**d. **Video Controller**
**`src/controllers/videoController.js`**
```javascript
// src/controllers/videoController.js
const videoService = require('../services/videoService');
const CustomError = require('../utils/CustomError');
exports.getVideo = async (req, res, next) => {
try {
const { courseId, videoKey } = req.params;
// Optionally, verify user has access to the course
const videoUrl = await videoService.getVideoUrl(courseId, videoKey);
res.status(200).json({
success: true,
data: {
url: videoUrl,
},
});
} catch (error) {
next(error);
}
};
```
**e. **Video Routes**
**`src/routes/v1/videoRoutes.js`**
```javascript
// src/routes/v1/videoRoutes.js
const express = require('express');
const router = express.Router();
const { getVideo } = require('../../controllers/videoController');
const { param } = require('express-validator');
const validate = require('../../middleware/validate');
const authMiddleware = require('../../middleware/authMiddleware');
/**
* @swagger
* tags:
* name: Videos
* description: Video streaming
*/
// Get Video URL
router.get(
'/:courseId/videos/:videoKey',
authMiddleware,
[
param('courseId').isUUID().withMessage('Valid courseId is required'),
param('videoKey').notEmpty().withMessage('videoKey is required'),
],
validate,
getVideo
);
module.exports = router;
```
**f. **Register Video Routes**
Add video routes to `app.js` or the appropriate routing file.
```javascript
// src/app.js
// ... existing imports
const videoRoutes = require('./routes/v1/videoRoutes');
app.use('/v1/videos', videoRoutes);
// ... existing code
```
---
#### 7. **Develop Quiz System**
Allow instructors to create quizzes and users to submit answers, with automatic grading.
**a. **Quiz Routes**
**`src/routes/v1/quizRoutes.js`**
```javascript
// src/routes/v1/quizRoutes.js
const express = require('express');
const router = express.Router();
const { createQuiz, getQuizById, submitQuiz } = require('../../controllers/quizController');
const { body, param } = require('express-validator');
const validate = require('../../middleware/validate');
const authMiddleware = require('../../middleware/authMiddleware');
const roleMiddleware = require('../../middleware/roleMiddleware');
/**
* @swagger
* tags:
* name: Quizzes
* description: Quiz management
*/
// Create Quiz
router.post(
'/',
authMiddleware,
roleMiddleware(['admin', 'instructor']),
[
body('courseId').isUUID().withMessage('Valid courseId is required'),
body('questions').isArray({ min: 1 }).withMessage('At least one question is required'),
body('questions.*.questionText').notEmpty().withMessage('Question text is required'),
body('questions.*.options').isArray({ min: 2 }).withMessage('At least two options are required'),
body('questions.*.correctAnswer').notEmpty().withMessage('Correct answer is required'),
],
validate,
createQuiz
);
// Get Quiz by ID
router.get(
'/:quizId',
[
param('quizId').isUUID().withMessage('Valid quizId is required'),
],
validate,
getQuizById
);
// Submit Quiz Answers
router.post(
'/:quizId/submit',
authMiddleware,
[
param('quizId').isUUID().withMessage('Valid quizId is required'),
body('answers').isObject().withMessage('Answers must be provided as an object'),
body('answers.*').notEmpty().withMessage('Each answer must have a selected option'),
],
validate,
submitQuiz
);
module.exports = router;
```
**b. **Quiz Controller**
**`src/controllers/quizController.js`**
```javascript
// src/controllers/quizController.js
const quizService = require('../services/quizService');
const CustomError = require('../utils/CustomError');
exports.createQuiz = async (req, res, next) => {
try {
const { courseId, questions } = req.body;
const instructorId = req.user.id;
// Create quiz
const quiz = await quizService.createQuiz({ courseId, questions, instructorId });
res.status(201).json({
success: true,
data: quiz,
});
} catch (error) {
next(error);
}
};
exports.getQuizById = async (req, res, next) => {
try {
const { quizId } = req.params;
const quiz = await quizService.getQuizById(quizId);
if (!quiz) {
throw new CustomError('Quiz not found', 404);
}
res.status(200).json({
success: true,
data: quiz,
});
} catch (error) {
next(error);
}
};
exports.submitQuiz = async (req, res, next) => {
try {
const { quizId } = req.params;
const { answers } = req.body;
const userId = req.user.id;
// Submit quiz
const quizResult = await quizService.submitQuiz(userId, quizId, answers);
res.status(200).json({
success: true,
data: quizResult,
});
} catch (error) {
next(error);
}
};
```
**c. **Quiz Service**
**`src/services/quizService.js`**
```javascript
// src/services/quizService.js
const quizRepository = require('../repositories/quizRepository');
const courseService = require('./courseService');
const userProgressService = require('./progressService');
const CustomError = require('../utils/CustomError');
const createQuiz = async ({ courseId, questions, instructorId }) => {
// Validate course ownership
const course = await courseService.getCourseById(courseId);
if (!course) {
throw new CustomError('Course not found', 404);
}
if (course.instructorId !== instructorId && !['admin'].some(role => role === 'admin')) {
throw new CustomError('Not authorized to add quiz to this course', 403);
}
// Create quiz
const quiz = await quizRepository.createQuiz({ courseId, questions });
return quiz;
};
const getQuizById = async (quizId) => {
return await quizRepository.findById(quizId);
};
const submitQuiz = async (userId, quizId, answers) => {
const quiz = await quizRepository.findById(quizId);
if (!quiz) {
throw new CustomError('Quiz not found', 404);
}
// Grade the quiz
let score = 0;
const correctAnswers = [];
quiz.questions.forEach((question, index) => {
const userAnswer = answers[`question${index + 1}`];
if (userAnswer && userAnswer === question.correctAnswer) {
score += 1;
correctAnswers.push({
questionId: question.id,
correctOption: question.correctAnswer,
});
}
});
const total = quiz.questions.length;
const percentageScore = (score / total) * 100;
// Update user progress
await userProgressService.updateUserProgress(userId, quiz.courseId, {
quizScores: { [`quiz${quizId}`]: percentageScore },
});
// Return results
return {
quizId,
score: percentageScore,
total,
correctAnswers,
};
};
module.exports = {
createQuiz,
getQuizById,
submitQuiz,
};
```
**d. **Quiz Repository**
**`src/repositories/quizRepository.js`**
```javascript
// src/repositories/quizRepository.js
const { Quiz } = require('../models');
class QuizRepository {
async createQuiz({ courseId, questions }) {
return await Quiz.create({ courseId, questions });
}
async findById(quizId) {
return await Quiz.findByPk(quizId, {
include: [{ model: require('./courseRepository').Course, as: 'course' }],
});
}
}
module.exports = new QuizRepository();
```
**e. **Mongoose Schema for Quizzes (Optional)**
If you decide to store quizzes in MongoDB, adjust the schema and repository accordingly. However, since quizzes are more structured, storing them in PostgreSQL is recommended.
---
#### 8. **Integrate Real-Time Features with WebSockets**
Enable real-time updates such as live notifications and progress tracking.
**a. **Set Up WebSocket Server**
Use **Socket.io** for handling WebSocket connections.
**Install Socket.io:**
```bash
npm install socket.io
```
**b. **Configure WebSocket in `index.js`**
Modify `index.js` to set up the Socket.io server.
```javascript
// src/index.js
require('dotenv').config();
const http = require('http');
const app = require('./app');
const { sequelize } = require('./models'); // Sequelize instance
const connectDB = require('../config/mongoose');
const { connectRedis } = require('../config/redis');
const logger = require('./utils/logger');
const { Server } = require('socket.io');
const PORT = process.env.PORT || 3000;
const httpServer = http.createServer(app);
const io = new Server(httpServer, {
cors: {
origin: ['https://www.educonnect.com', 'https://app.educonnect.com'], // Allowed origins
methods: ['GET', 'POST'],
},
});
// Middleware for authenticating WebSocket connections
io.use(async (socket, next) => {
try {
const token = socket.handshake.auth.token;
if (!token) {
return next(new Error('Authentication error'));
}
const decoded = jwt.verify(token, process.env.JWT_SECRET);
socket.user = {
id: decoded.id,
roles: decoded.roles,
};
next();
} catch (error) {
next(new Error('Authentication error'));
}
});
// Handle WebSocket connections
io.on('connection', (socket) => {
logger.info(`User connected: ${socket.user.id}`);
// Join rooms based on user or course
socket.join(`user_${socket.user.id}`);
// Additional room logic as needed
// Handle disconnection
socket.on('disconnect', () => {
logger.info(`User disconnected: ${socket.user.id}`);
});
});
// Export io for use in services/controllers
module.exports = { io };
const startServer = async () => {
try {
// Connect to PostgreSQL
await sequelize.authenticate();
logger.info('PostgreSQL connected successfully.');
// Connect to MongoDB
await connectDB();
// Connect to Redis
await connectRedis();
// Start the server
httpServer.listen(PORT, () => {
logger.info(`Server is running on port ${PORT}`);
});
} catch (error) {
logger.error('Database connection failed:', error);
process.exit(1);
}
};
startServer();
```
**c. **Emit Events from Services**
Trigger real-time events from your services whenever certain actions occur.
**Example: Notify Users When a New Course is Created**
**Modify Course Service:**
```javascript
// src/services/courseService.js
const courseRepository = require('../repositories/courseRepository');
const learningMaterialService = require('./learningMaterialService');
const quizService = require('./quizService');
const { io } = require('../index'); // Import the Socket.io instance
const createCourse = async (courseData) => {
// Create course
const course = await courseRepository.create(courseData);
// Emit event to notify users
io.emit('NEW_COURSE', {
courseId: course.id,
title: course.title,
description: course.description,
category: course.category,
});
return course;
};
// ... other methods
module.exports = {
createCourse,
// ... other exports
};
```
**Example: Real-Time Progress Update**
**Modify Progress Service:**
```javascript
// src/services/progressService.js
const userProgressRepository = require('../repositories/userProgressRepository');
const courseService = require('./courseService');
const { io } = require('../index'); // Import the Socket.io instance
const getUserProgress = async (userId) => {
return await userProgressRepository.findByUserId(userId);
};
const updateUserProgress = async (userId, { courseId, completedModules, quizScores }) => {
// Validate course existence
const course = await courseService.getCourseById(courseId);
if (!course) {
throw new CustomError('Course not found', 404);
}
// Update progress
const progress = await userProgressRepository.updateProgress(userId, courseId, { completedModules, quizScores });
// Emit real-time update to the user
io.to(`user_${userId}`).emit('PROGRESS_UPDATE', {
courseId,
completedModules: progress.completedModules,
quizScores: progress.quizScores,
});
return progress;
};
module.exports = {
getUserProgress,
updateUserProgress,
};
```
**d. **Client-Side Integration**
Ensure that your frontend clients connect to the WebSocket server and handle incoming events appropriately.
**Example using Socket.io Client:**
```javascript
// frontend/src/socket.js
import { io } from 'socket.io-client';
const socket = io('https://api.educonnect.com', {
auth: {
token: 'JWT_TOKEN', // Replace with actual token
},
});
socket.on('connect', () => {
console.log('Connected to WebSocket server');
});
socket.on('NEW_COURSE', (data) => {
// Handle new course notification
console.log('New Course Available:', data);
});
socket.on('PROGRESS_UPDATE', (data) => {
// Handle progress update
console.log('Progress Updated:', data);
});
socket.on('disconnect', () => {
console.log('Disconnected from WebSocket server');
});
export default socket;
```
---
#### 9. **Implement Video Streaming Integration**
Integrate video streaming services to deliver course videos seamlessly.
**a. **Use Third-Party Video Platforms**
Consider integrating with platforms like **Vimeo**, **YouTube**, or **AWS Elemental Media Services** for scalable video streaming.
**Example: Embedding Vimeo Videos**
1. **Upload Videos to Vimeo:**
- Use Vimeo’s API to upload and manage videos programmatically.
2. **Store Vimeo Video URLs in Learning Materials:**
- When creating learning materials, include Vimeo embed URLs.
3. **Frontend Integration:**
- Use Vimeo’s embed player in your frontend application to stream videos.
**b. **Implement Video Upload and Management (Optional)**
If you choose to manage video uploads within your application:
1. **Set Up File Uploads:**
- Use libraries like **Multer** for handling multipart/form-data.
**Install Multer:**
```bash
npm install multer
```
2. **Create Video Upload Controller:**
**`src/controllers/videoUploadController.js`**
```javascript
// src/controllers/videoUploadController.js
const multer = require('multer');
const path = require('path');
const videoService = require('../services/videoService');
const CustomError = require('../utils/CustomError');
// Configure Multer Storage
const storage = multer.diskStorage({
destination: function (req, file, cb) {
cb(null, 'uploads/videos/');
},
filename: function (req, file, cb) {
cb(null, `${Date.now()}-${file.originalname}`);
},
});
// File Filter
const fileFilter = (req, file, cb) => {
const fileTypes = /mp4|avi|mkv/;
const extname = fileTypes.test(path.extname(file.originalname).toLowerCase());
const mimetype = fileTypes.test(file.mimetype);
if (mimetype && extname) {
return cb(null, true);
} else {
cb(new CustomError('Only video files are allowed', 400));
}
};
const upload = multer({
storage,
limits: { fileSize: 100 * 1024 * 1024 }, // 100MB limit
fileFilter,
});
// Video Upload Handler
exports.uploadVideo = [
upload.single('video'),
async (req, res, next) => {
try {
if (!req.file) {
throw new CustomError('No file uploaded', 400);
}
// Process and store video (e.g., upload to S3)
const videoUrl = await videoService.uploadVideoToS3(req.file);
res.status(201).json({
success: true,
data: {
url: videoUrl,
},
});
} catch (error) {
next(error);
}
},
];
```
3. **Video Upload Service**
**`src/services/videoService.js`**
```javascript
// src/services/videoService.js
const AWS = require('aws-sdk');
const fs = require('fs');
const path = require('path');
const CustomError = require('../utils/CustomError');
const s3 = new AWS.S3({
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
region: process.env.AWS_REGION,
});
const uploadVideoToS3 = async (file) => {
const fileContent = fs.readFileSync(file.path);
const params = {
Bucket: process.env.AWS_S3_BUCKET,
Key: `videos/${Date.now()}-${file.originalname}`,
Body: fileContent,
ContentType: file.mimetype,
};
try {
const data = await s3.upload(params).promise();
// Optionally, delete the local file after upload
fs.unlinkSync(file.path);
return data.Location; // URL of the uploaded video
} catch (error) {
throw new CustomError('Failed to upload video', 500);
}
};
module.exports = {
uploadVideoToS3,
};
```
4. **Video Upload Routes**
**`src/routes/v1/videoUploadRoutes.js`**
```javascript
// src/routes/v1/videoUploadRoutes.js
const express = require('express');
const router = express.Router();
const { uploadVideo } = require('../../controllers/videoUploadController');
const authMiddleware = require('../../middleware/authMiddleware');
const roleMiddleware = require('../../middleware/roleMiddleware');
/**
* @swagger
* tags:
* name: Video Upload
* description: Upload and manage course videos
*/
// Upload Video
router.post(
'/upload',
authMiddleware,
roleMiddleware(['admin', 'instructor']),
uploadVideo
);
module.exports = router;
```
5. **Register Video Upload Routes**
Add video upload routes to `app.js` or the appropriate routing file.
```javascript
// src/app.js
// ... existing imports
const videoUploadRoutes = require('./routes/v1/videoUploadRoutes');
app.use('/v1/video', videoUploadRoutes);
// ... existing code
```
---
#### 10. **Implement Quiz System Enhancements**
Enhance the quiz system with features like timed quizzes, question types, and feedback.
**a. **Add Timed Quizzes**
1. **Update Quiz Schema:**
**`models/quiz.js`**
```javascript
// models/quiz.js
'use strict';
const { Model } = require('sequelize');
module.exports = (sequelize, DataTypes) => {
class Quiz extends Model {
static associate(models) {
// Define associations here
Quiz.belongsTo(models.Course, { foreignKey: 'courseId', as: 'course' });
}
}
Quiz.init(
{
title: {
type: DataTypes.STRING,
allowNull: false
},
courseId: {
type: DataTypes.UUID,
allowNull: false,
references: {
model: 'Courses',
key: 'id'
}
},
questions: {
type: DataTypes.JSONB,
allowNull: false
},
timeLimit: { // Time limit in minutes
type: DataTypes.INTEGER,
allowNull: true
}
},
{
sequelize,
modelName: 'Quiz',
tableName: 'Quizzes',
timestamps: true
}
);
return Quiz;
};
```
2. **Create Migration for `timeLimit`:**
```bash
npx sequelize-cli migration:generate --name add-timeLimit-to-quizzes
```
**`migrations/XXXXXX-add-timeLimit-to-quizzes.js`:**
```javascript
'use strict';
module.exports = {
up: async (queryInterface, Sequelize) => {
await queryInterface.addColumn('Quizzes', 'timeLimit', {
type: Sequelize.INTEGER,
allowNull: true,
});
},
down: async (queryInterface, Sequelize) => {
await queryInterface.removeColumn('Quizzes', 'timeLimit');
}
};
```
**Run Migration:**
```bash
npx sequelize-cli db:migrate
```
3. **Update Quiz Controller to Handle `timeLimit`:**
**`src/controllers/quizController.js`**
```javascript
// Modify createQuiz to include timeLimit
exports.createQuiz = async (req, res, next) => {
try {
const { courseId, questions, timeLimit } = req.body;
const instructorId = req.user.id;
// Create quiz
const quiz = await quizService.createQuiz({ courseId, questions, timeLimit, instructorId });
res.status(201).json({
success: true,
data: quiz,
});
} catch (error) {
next(error);
}
};
```
4. **Update Quiz Service to Handle `timeLimit`:**
**`src/services/quizService.js`**
```javascript
const createQuiz = async ({ courseId, questions, timeLimit, instructorId }) => {
// Validate course ownership
const course = await courseService.getCourseById(courseId);
if (!course) {
throw new CustomError('Course not found', 404);
}
if (course.instructorId !== instructorId && !['admin'].includes('admin')) {
throw new CustomError('Not authorized to add quiz to this course', 403);
}
// Create quiz
const quiz = await quizRepository.createQuiz({ courseId, questions, timeLimit });
return quiz;
};
```
5. **Handle Timed Quizzes on the Frontend:**
Implement countdown timers and enforce time limits when users take quizzes. Notify users when time expires and automatically submit their answers.
---
#### 11. **Implement Comprehensive Testing**
Ensure the reliability and correctness of your backend by writing and running comprehensive tests.
**a. **Unit Tests**
Write unit tests for individual components like services and repositories.
**Example: Testing Course Service**
**`tests/unit/courseService.test.js`**
```javascript
// tests/unit/courseService.test.js
const courseService = require('../../src/services/courseService');
const courseRepository = require('../../src/repositories/courseRepository');
const { Course } = require('../../src/models');
const CustomError = require('../../src/utils/CustomError');
jest.mock('../../src/repositories/courseRepository');
describe('Course Service', () => {
beforeEach(() => {
jest.clearAllMocks();
});
it('should create a course successfully', async () => {
const courseData = {
title: 'Test Course',
description: 'Course Description',
category: 'Testing',
videos: ['video1.mp4'],
materials: ['material1.pdf'],
instructorId: 'instructor-uuid',
};
courseRepository.create.mockResolvedValue({ id: 'course-uuid', ...courseData });
const course = await courseService.createCourse(courseData);
expect(course).toHaveProperty('id', 'course-uuid');
expect(course.title).toBe('Test Course');
expect(courseRepository.create).toHaveBeenCalledWith(courseData);
});
it('should throw an error if course does not exist', async () => {
courseService.getCourseById = jest.fn().mockResolvedValue(null);
const courseData = {
title: 'Test Course',
description: 'Course Description',
category: 'Testing',
videos: ['video1.mp4'],
materials: ['material1.pdf'],
instructorId: 'instructor-uuid',
};
await expect(courseService.createCourse(courseData)).rejects.toThrow(CustomError);
});
// Add more tests as needed
});
```
**b. **Integration Tests**
Test the interaction between different modules, such as controllers and services.
**Example: Testing User Registration Endpoint**
**`tests/integration/userRegistration.test.js`**
```javascript
// tests/integration/userRegistration.test.js
const request = require('supertest');
const app = require('../../src/app');
const { sequelize } = require('../../src/models');
const bcrypt = require('bcrypt');
beforeAll(async () => {
await sequelize.sync({ force: true });
});
afterAll(async () => {
await sequelize.close();
});
describe('User Registration', () => {
it('should register a new user', async () => {
const res = await request(app)
.post('/v1/users/register')
.send({
username: 'testuser',
email: 'testuser@example.com',
password: 'password123',
});
expect(res.statusCode).toEqual(201);
expect(res.body).toHaveProperty('success', true);
expect(res.body.data).toHaveProperty('id');
expect(res.body.data).toHaveProperty('token');
expect(res.body.data).toHaveProperty('refreshToken');
});
it('should not register a user with existing email', async () => {
// Attempt to register with the same email
const res = await request(app)
.post('/v1/users/register')
.send({
username: 'testuser2',
email: 'testuser@example.com',
password: 'password123',
});
expect(res.statusCode).toEqual(400);
expect(res.body).toHaveProperty('success', false);
expect(res.body).toHaveProperty('message', 'User already exists with this email');
});
// Add more tests as needed
});
```
**c. **End-to-End (E2E) Tests**
Simulate real user scenarios to test the entire application flow.
**Example: Using Cypress for E2E Testing**
1. **Install Cypress:**
```bash
npm install --save-dev cypress
```
2. **Initialize Cypress:**
```bash
npx cypress open
```
3. **Write E2E Tests:**
**`cypress/integration/userFlow.spec.js`**
```javascript
// cypress/integration/userFlow.spec.js
describe('User Registration and Login', () => {
it('should register and login a user', () => {
// Register User
cy.request('POST', 'https://api.educonnect.com/v1/users/register', {
username: 'e2etestuser',
email: 'e2etestuser@example.com',
password: 'password123',
}).then((response) => {
expect(response.status).to.eq(201);
expect(response.body.data).to.have.property('token');
const token = response.body.data.token;
// Login User
cy.request('POST', 'https://api.educonnect.com/v1/users/login', {
email: 'e2etestuser@example.com',
password: 'password123',
}).then((loginResponse) => {
expect(loginResponse.status).to.eq(200);
expect(loginResponse.body.data).to.have.property('token');
});
});
});
});
```
4. **Run Cypress Tests:**
```bash
npx cypress run
```
---
#### 12. **Summary**
In **Step 4: Implement Core Functionality**, we've:
- **Set Up Express Middleware and Routing:** Organized the application structure for scalability and maintainability.
- **Developed User Authentication and Management:** Enabled secure user registration, login, profile management, and role-based access control.
- **Implemented Course Management:** Allowed instructors/admins to create, read, update, and delete courses.
- **Set Up Progress Tracking:** Enabled users to track their progress through courses, including completed modules and quiz scores.
- **Integrated Adaptive Learning Algorithms:** Adjusted learning paths based on user performance to enhance the learning experience.
- **Implemented Video Streaming Integration:** Facilitated seamless video content delivery using third-party platforms like AWS S3 and CloudFront.
- **Developed Quiz System:** Allowed creation and submission of quizzes with automatic grading and feedback.
- **Integrated Real-Time Features with WebSockets:** Enabled live notifications and real-time progress updates using Socket.io.
- **Enhanced Quiz System:** Added features like timed quizzes to improve the assessment process.
- **Implemented Comprehensive Testing:** Ensured backend reliability through unit, integration, and end-to-end testing.
---
### Next Steps
With the core functionalities implemented, the next phase is **Step 5: Authentication and Authorization**. In this step, we'll refine the authentication mechanisms, manage user roles and permissions, and ensure that only authorized users can access specific resources.
Feel free to let me know if you need further details on any of the implemented features or if you're ready to proceed to the next step!
====
### Step 5: Authentication and Authorization
In **Step 5: Authentication and Authorization**, we will enhance and secure the **EduConnect: Adaptive Learning Backend** by implementing robust authentication mechanisms and fine-grained authorization controls. This step ensures that only authenticated users can access the system and that they have appropriate permissions based on their roles. We will cover:
1. **Reviewing Existing Authentication Setup**
2. **Implementing Role-Based Access Control (RBAC)**
3. **Integrating OAuth2 for Social Logins**
4. **Adding Account Verification and Password Reset Functionality**
5. **Securing Token Management with Refresh Tokens**
6. **Enhancing Authorization Middleware**
7. **Adhering to Security Best Practices**
8. **Comprehensive Testing of Authentication and Authorization**
---
#### 1. **Reviewing Existing Authentication Setup**
Before making enhancements, it's crucial to understand the current authentication implementation. From **Step 4**, we've established:
- **User Registration and Login:** Users can register and log in, receiving JWT access and refresh tokens.
- **JWT-Based Authentication Middleware:** Protects routes by verifying JWT tokens.
- **Role-Based Access Control (RBAC):** Basic role checks (e.g., 'admin', 'instructor') to restrict access to certain endpoints.
**Objective:** Strengthen and expand these capabilities to ensure security, flexibility, and scalability.
---
#### 2. **Implementing Role-Based Access Control (RBAC)**
RBAC allows us to assign permissions to users based on their roles, ensuring that users can only perform actions permitted by their roles.
**a. **Define User Roles and Permissions**
First, clearly define the roles and their associated permissions.
**Example Roles:**
- **Student:** Access to courses, learning materials, submit quizzes.
- **Instructor:** Create and manage courses, quizzes, view student progress.
- **Admin:** Full access, including user management, system settings.
**b. **Update User Model to Support Roles**
Ensure the `User` model can handle multiple roles.
**`models/user.js`:**
```javascript
// models/user.js
// ... existing imports and code
User.init(
{
// ... existing fields
roles: {
type: DataTypes.ARRAY(DataTypes.STRING),
allowNull: false,
defaultValue: ['student'],
validate: {
isValidRole(value) {
const validRoles = ['student', 'instructor', 'admin'];
value.forEach(role => {
if (!validRoles.includes(role)) {
throw new Error(`Invalid role: ${role}`);
}
});
},
},
},
},
{
sequelize,
modelName: 'User',
tableName: 'Users',
timestamps: true,
paranoid: true, // From Step 3: Soft deletes
}
);
// ... existing associations and exports
```
**c. **Create Authorization Middleware**
Develop middleware to check if a user has the required role(s) to access a route.
**`src/middleware/authorize.js`:**
```javascript
// src/middleware/authorize.js
const CustomError = require('../utils/CustomError');
/**
* Middleware to authorize based on user roles.
* @param {Array} roles - Array of roles permitted to access the route.
*/
const authorize = (roles = []) => {
// roles param can be a single role string (e.g., 'admin') or an array of roles
if (typeof roles === 'string') {
roles = [roles];
}
return (req, res, next) => {
if (!req.user || !req.user.roles) {
return next(new CustomError('Access denied: No roles found', 403));
}
const hasRole = req.user.roles.some(role => roles.includes(role));
if (!hasRole) {
return next(new CustomError('Access denied: Insufficient permissions', 403));
}
next();
};
};
module.exports = authorize;
```
**d. **Apply Authorization Middleware to Routes**
Use the `authorize` middleware in routes to enforce role-based access.
**Example: Protecting Course Creation Route for Instructors and Admins**
**`src/routes/v1/courseRoutes.js`:**
```javascript
// src/routes/v1/courseRoutes.js
// ... existing imports
const authorize = require('../../middleware/authorize');
// Create Course
router.post(
'/',
authMiddleware,
authorize(['admin', 'instructor']),
[
body('title').notEmpty().withMessage('Title is required'),
body('description').notEmpty().withMessage('Description is required'),
body('category').notEmpty().withMessage('Category is required'),
body('videos').isArray().withMessage('Videos should be an array of URLs'),
body('materials').isArray().withMessage('Materials should be an array of URLs'),
],
validate,
createCourse
);
// ... other routes
```
**e. **Example: Protecting Admin-Only Routes**
**`src/routes/v1/adminRoutes.js`:**
```javascript
// src/routes/v1/adminRoutes.js
const express = require('express');
const router = express.Router();
const { getAllUsers, deleteUserById, updateUserRoles } = require('../../controllers/adminController');
const { param, body } = require('express-validator');
const validate = require('../../middleware/validate');
const authMiddleware = require('../../middleware/authMiddleware');
const authorize = require('../../middleware/authorize');
/**
* @swagger
* tags:
* name: Admin
* description: Admin-specific operations
*/
// Get All Users
router.get(
'/users',
authMiddleware,
authorize('admin'),
getAllUsers
);
// Update User Roles
router.put(
'/users/:userId/roles',
authMiddleware,
authorize('admin'),
[
param('userId').isUUID().withMessage('Valid userId is required'),
body('roles').isArray().withMessage('Roles must be an array of valid role strings'),
body('roles.*').isIn(['student', 'instructor', 'admin']).withMessage('Invalid role provided'),
],
validate,
updateUserRoles
);
// Delete User
router.delete(
'/users/:userId',
authMiddleware,
authorize('admin'),
[
param('userId').isUUID().withMessage('Valid userId is required'),
],
validate,
deleteUserById
);
module.exports = router;
```
**f. **Admin Controller and Service**
Implement controllers and services to handle admin operations like managing users.
**`src/controllers/adminController.js`:**
```javascript
// src/controllers/adminController.js
const adminService = require('../services/adminService');
const CustomError = require('../utils/CustomError');
// Get All Users
exports.getAllUsers = async (req, res, next) => {
try {
const users = await adminService.getAllUsers();
res.status(200).json({
success: true,
data: users,
});
} catch (error) {
next(error);
}
};
// Update User Roles
exports.updateUserRoles = async (req, res, next) => {
try {
const { userId } = req.params;
const { roles } = req.body;
const updatedUser = await adminService.updateUserRoles(userId, roles);
if (!updatedUser) {
throw new CustomError('User not found', 404);
}
res.status(200).json({
success: true,
data: updatedUser,
});
} catch (error) {
next(error);
}
};
// Delete User by ID
exports.deleteUserById = async (req, res, next) => {
try {
const { userId } = req.params;
const success = await adminService.deleteUserById(userId);
if (!success) {
throw new CustomError('User not found', 404);
}
res.status(200).json({
success: true,
message: 'User deleted successfully',
});
} catch (error) {
next(error);
}
};
```
**`src/services/adminService.js`:**
```javascript
// src/services/adminService.js
const userRepository = require('../repositories/userRepository');
const getAllUsers = async () => {
return await userRepository.findAllUsers();
};
const updateUserRoles = async (userId, roles) => {
return await userRepository.updateRoles(userId, roles);
};
const deleteUserById = async (userId) => {
return await userRepository.delete(userId);
};
module.exports = {
getAllUsers,
updateUserRoles,
deleteUserById,
};
```
**`src/repositories/userRepository.js`:**
```javascript
// src/repositories/userRepository.js
const { User } = require('../models');
class UserRepository {
// ... existing methods
async findAllUsers() {
return await User.findAll({
attributes: ['id', 'username', 'email', 'roles', 'createdAt', 'updatedAt'],
});
}
async updateRoles(userId, roles) {
const user = await User.findByPk(userId);
if (!user) return null;
await user.update({ roles });
return user;
}
}
module.exports = new UserRepository();
```
**g. **Register Admin Routes**
Add admin routes to your application.
**`src/app.js`:**
```javascript
// src/app.js
// ... existing imports
const adminRoutes = require('./routes/v1/adminRoutes');
app.use('/v1/admin', adminRoutes);
// ... existing code
```
---
#### 3. **Integrating OAuth2 for Social Logins**
Enhance user experience by allowing users to log in using third-party providers like Google, Facebook, or GitHub. We'll use **Passport.js** to facilitate OAuth2 integration.
**a. **Install Necessary Packages**
```bash
npm install passport passport-google-oauth20 passport-facebook passport-github2
```
**b. **Configure Passport Strategies**
Set up Passport strategies for each provider.
**`src/config/passport.js`:**
```javascript
// src/config/passport.js
const passport = require('passport');
const GoogleStrategy = require('passport-google-oauth20').Strategy;
const FacebookStrategy = require('passport-facebook').Strategy;
const GitHubStrategy = require('passport-github2').Strategy;
const userService = require('../services/userService');
const CustomError = require('../utils/CustomError');
// Serialize and Deserialize User (Optional if using sessions)
passport.serializeUser((user, done) => {
done(null, user.id);
});
passport.deserializeUser(async (id, done) => {
try {
const user = await userService.findUserById(id);
done(null, user);
} catch (error) {
done(error, null);
}
});
// Google OAuth Strategy
passport.use(
new GoogleStrategy(
{
clientID: process.env.GOOGLE_CLIENT_ID,
clientSecret: process.env.GOOGLE_CLIENT_SECRET,
callbackURL: '/v1/auth/google/callback',
},
async (accessToken, refreshToken, profile, done) => {
try {
// Extract user information
const email = profile.emails[0].value;
const username = profile.displayName;
// Check if user exists
let user = await userService.findUserByEmail(email);
if (!user) {
// Create new user
user = await userService.createUser({
username,
email,
password: '', // No password since using OAuth
roles: ['student'],
});
}
done(null, user);
} catch (error) {
done(error, null);
}
}
)
);
// Facebook OAuth Strategy
passport.use(
new FacebookStrategy(
{
clientID: process.env.FACEBOOK_APP_ID,
clientSecret: process.env.FACEBOOK_APP_SECRET,
callbackURL: '/v1/auth/facebook/callback',
profileFields: ['id', 'displayName', 'emails'],
},
async (accessToken, refreshToken, profile, done) => {
try {
const email = profile.emails[0].value;
const username = profile.displayName;
let user = await userService.findUserByEmail(email);
if (!user) {
user = await userService.createUser({
username,
email,
password: '',
roles: ['student'],
});
}
done(null, user);
} catch (error) {
done(error, null);
}
}
)
);
// GitHub OAuth Strategy
passport.use(
new GitHubStrategy(
{
clientID: process.env.GITHUB_CLIENT_ID,
clientSecret: process.env.GITHUB_CLIENT_SECRET,
callbackURL: '/v1/auth/github/callback',
},
async (accessToken, refreshToken, profile, done) => {
try {
const email = profile.emails[0].value;
const username = profile.username;
let user = await userService.findUserByEmail(email);
if (!user) {
user = await userService.createUser({
username,
email,
password: '',
roles: ['student'],
});
}
done(null, user);
} catch (error) {
done(error, null);
}
}
)
);
module.exports = passport;
```
**c. **Set Up OAuth Routes**
Create routes to initiate and handle callbacks from OAuth providers.
**`src/routes/v1/oauthRoutes.js`:**
```javascript
// src/routes/v1/oauthRoutes.js
const express = require('express');
const router = express.Router();
const passport = require('../../config/passport');
const jwt = require('jsonwebtoken');
const CustomError = require('../../utils/CustomError');
/**
* @swagger
* tags:
* name: OAuth
* description: OAuth authentication routes
*/
// Google OAuth
router.get('/google', passport.authenticate('google', { scope: ['profile', 'email'] }));
router.get(
'/google/callback',
passport.authenticate('google', { session: false, failureRedirect: '/login' }),
(req, res) => {
// Successful authentication, issue JWT
const token = jwt.sign(
{ id: req.user.id, roles: req.user.roles },
process.env.JWT_SECRET,
{ expiresIn: '1h' }
);
const refreshToken = jwt.sign(
{ id: req.user.id },
process.env.JWT_REFRESH_SECRET,
{ expiresIn: '7d' }
);
// Redirect or respond with tokens
res.status(200).json({
success: true,
data: {
token,
refreshToken,
},
});
}
);
// Facebook OAuth
router.get('/facebook', passport.authenticate('facebook', { scope: ['email'] }));
router.get(
'/facebook/callback',
passport.authenticate('facebook', { session: false, failureRedirect: '/login' }),
(req, res) => {
const token = jwt.sign(
{ id: req.user.id, roles: req.user.roles },
process.env.JWT_SECRET,
{ expiresIn: '1h' }
);
const refreshToken = jwt.sign(
{ id: req.user.id },
process.env.JWT_REFRESH_SECRET,
{ expiresIn: '7d' }
);
res.status(200).json({
success: true,
data: {
token,
refreshToken,
},
});
}
);
// GitHub OAuth
router.get('/github', passport.authenticate('github', { scope: ['user:email'] }));
router.get(
'/github/callback',
passport.authenticate('github', { session: false, failureRedirect: '/login' }),
(req, res) => {
const token = jwt.sign(
{ id: req.user.id, roles: req.user.roles },
process.env.JWT_SECRET,
{ expiresIn: '1h' }
);
const refreshToken = jwt.sign(
{ id: req.user.id },
process.env.JWT_REFRESH_SECRET,
{ expiresIn: '7d' }
);
res.status(200).json({
success: true,
data: {
token,
refreshToken,
},
});
}
);
module.exports = router;
```
**d. **Register OAuth Routes**
Add OAuth routes to your application.
**`src/app.js`:**
```javascript
// src/app.js
// ... existing imports
const oauthRoutes = require('./routes/v1/oauthRoutes');
app.use('/v1/auth', oauthRoutes);
// ... existing code
```
**e. **Update Application Entry Point for Passport**
Initialize Passport in your application.
**`src/index.js`:**
```javascript
// src/index.js
require('dotenv').config();
const http = require('http');
const app = require('./app');
const { sequelize } = require('./models'); // Sequelize instance
const connectDB = require('../config/mongoose');
const { connectRedis } = require('../config/redis');
const logger = require('./utils/logger');
const { Server } = require('socket.io');
const passport = require('./config/passport'); // Import Passport configuration
const PORT = process.env.PORT || 3000;
const httpServer = http.createServer(app);
// Initialize Passport
app.use(passport.initialize());
// ... existing WebSocket setup
const startServer = async () => {
try {
// Connect to PostgreSQL
await sequelize.authenticate();
logger.info('PostgreSQL connected successfully.');
// Connect to MongoDB
await connectDB();
// Connect to Redis
await connectRedis();
// Start the server
httpServer.listen(PORT, () => {
logger.info(`Server is running on port ${PORT}`);
});
} catch (error) {
logger.error('Database connection failed:', error);
process.exit(1);
}
};
startServer();
```
**f. **Configure Environment Variables**
Add OAuth credentials to your `.env` file.
**`.env`:**
```env
# OAuth Credentials
# Google
GOOGLE_CLIENT_ID=your_google_client_id
GOOGLE_CLIENT_SECRET=your_google_client_secret
# Facebook
FACEBOOK_APP_ID=your_facebook_app_id
FACEBOOK_APP_SECRET=your_facebook_app_secret
# GitHub
GITHUB_CLIENT_ID=your_github_client_id
GITHUB_CLIENT_SECRET=your_github_client_secret
```
**g. **Secure OAuth Routes**
Ensure that OAuth callbacks and tokens are securely handled. Use HTTPS in production to encrypt data in transit.
---
#### 4. **Adding Account Verification and Password Reset Functionality**
Enhance security and user trust by implementing account verification (email confirmation) and password reset mechanisms.
**a. **Account Verification (Email Confirmation)**
**i. **Modify User Model**
Add `isVerified` and `verificationToken` fields to the `User` model.
**`models/user.js`:**
```javascript
// models/user.js
// ... existing imports and code
User.init(
{
// ... existing fields
isVerified: {
type: DataTypes.BOOLEAN,
defaultValue: false,
},
verificationToken: {
type: DataTypes.STRING,
allowNull: true,
},
},
{
sequelize,
modelName: 'User',
tableName: 'Users',
timestamps: true,
paranoid: true,
}
);
// ... existing associations and exports
```
**ii. **Create Migration for New Fields**
```bash
npx sequelize-cli migration:generate --name add-verification-fields-to-users
```
**`migrations/XXXXXX-add-verification-fields-to-users.js`:**
```javascript
'use strict';
module.exports = {
up: async (queryInterface, Sequelize) => {
await queryInterface.addColumn('Users', 'isVerified', {
type: Sequelize.BOOLEAN,
defaultValue: false,
});
await queryInterface.addColumn('Users', 'verificationToken', {
type: Sequelize.STRING,
allowNull: true,
});
},
down: async (queryInterface, Sequelize) => {
await queryInterface.removeColumn('Users', 'verificationToken');
await queryInterface.removeColumn('Users', 'isVerified');
}
};
```
**Run Migration:**
```bash
npx sequelize-cli db:migrate
```
**iii. **Send Verification Email Upon Registration**
Use a library like **Nodemailer** to send verification emails.
**Install Nodemailer:**
```bash
npm install nodemailer
```
**Create Email Service:**
**`src/services/emailService.js`:**
```javascript
// src/services/emailService.js
const nodemailer = require('nodemailer');
const CustomError = require('../utils/CustomError');
const transporter = nodemailer.createTransport({
host: process.env.EMAIL_HOST, // e.g., smtp.gmail.com
port: process.env.EMAIL_PORT, // e.g., 587
secure: false, // true for 465, false for other ports
auth: {
user: process.env.EMAIL_USER,
pass: process.env.EMAIL_PASS,
},
});
const sendVerificationEmail = async (user, verificationToken) => {
const verificationUrl = `${process.env.FRONTEND_URL}/verify-email?token=${verificationToken}`;
const mailOptions = {
from: `"EduConnect Support" <${process.env.EMAIL_USER}>`,
to: user.email,
subject: 'Verify Your EduConnect Account',
html: `
<p>Hello ${user.username},</p>
<p>Thank you for registering at EduConnect. Please verify your email by clicking the link below:</p>
<a href="${verificationUrl}">Verify Email</a>
<p>If you did not register, please ignore this email.</p>
`,
};
try {
await transporter.sendMail(mailOptions);
} catch (error) {
throw new CustomError('Failed to send verification email', 500);
}
};
const sendPasswordResetEmail = async (user, resetToken) => {
const resetUrl = `${process.env.FRONTEND_URL}/reset-password?token=${resetToken}`;
const mailOptions = {
from: `"EduConnect Support" <${process.env.EMAIL_USER}>`,
to: user.email,
subject: 'Reset Your EduConnect Password',
html: `
<p>Hello ${user.username},</p>
<p>You requested to reset your password. Click the link below to proceed:</p>
<a href="${resetUrl}">Reset Password</a>
<p>If you did not request this, please ignore this email.</p>
`,
};
try {
await transporter.sendMail(mailOptions);
} catch (error) {
throw new CustomError('Failed to send password reset email', 500);
}
};
module.exports = {
sendVerificationEmail,
sendPasswordResetEmail,
};
```
**iv. **Update Registration Controller to Send Verification Email**
**`src/controllers/userController.js`:**
```javascript
// src/controllers/userController.js
const { v4: uuidv4 } = require('uuid');
const emailService = require('../services/emailService');
// Register User
exports.register = async (req, res, next) => {
try {
const { username, email, password } = req.body;
// Check if user exists
const existingUser = await userService.findUserByEmail(email);
if (existingUser) {
throw new CustomError('User already exists with this email', 400);
}
// Hash password
const hashedPassword = await bcrypt.hash(password, 10);
// Generate verification token
const verificationToken = uuidv4();
// Create user
const user = await userService.createUser({
username,
email,
password: hashedPassword,
roles: ['student'],
verificationToken,
});
// Send verification email
await emailService.sendVerificationEmail(user, verificationToken);
// Generate tokens
const token = generateToken(user);
const refreshToken = generateRefreshToken(user);
res.status(201).json({
success: true,
message: 'User registered successfully. Please verify your email.',
data: {
id: user.id,
username: user.username,
email: user.email,
token,
refreshToken,
},
});
} catch (error) {
next(error);
}
};
```
**v. **Create Verification Endpoint**
Allow users to verify their email addresses.
**`src/routes/v1/userRoutes.js`:**
```javascript
// src/routes/v1/userRoutes.js
// ... existing imports and routes
const { verifyEmail } = require('../../controllers/userController');
// Email Verification
router.get('/verify-email', async (req, res, next) => {
try {
const { token } = req.query;
if (!token) {
throw new CustomError('Verification token is missing', 400);
}
// Verify token
const user = await userService.findUserByVerificationToken(token);
if (!user) {
throw new CustomError('Invalid or expired verification token', 400);
}
// Update user verification status
await userService.verifyUser(user.id);
res.status(200).json({
success: true,
message: 'Email verified successfully. You can now log in.',
});
} catch (error) {
next(error);
}
});
```
**`src/controllers/userController.js`:**
```javascript
// src/controllers/userController.js
// ... existing imports and code
// Verify Email
exports.verifyEmail = async (req, res, next) => {
try {
const { token } = req.query;
if (!token) {
throw new CustomError('Verification token is missing', 400);
}
// Find user by verification token
const user = await userService.findUserByVerificationToken(token);
if (!user) {
throw new CustomError('Invalid or expired verification token', 400);
}
// Update user verification status
await userService.verifyUser(user.id);
res.status(200).json({
success: true,
message: 'Email verified successfully. You can now log in.',
});
} catch (error) {
next(error);
}
};
// ... existing code
```
**`src/services/userService.js`:**
```javascript
// src/services/userService.js
// ... existing imports and code
const findUserByVerificationToken = async (token) => {
return await userRepository.findByVerificationToken(token);
};
const verifyUser = async (userId) => {
return await userRepository.verifyUser(userId);
};
module.exports = {
// ... existing exports
findUserByVerificationToken,
verifyUser,
};
```
**`src/repositories/userRepository.js`:**
```javascript
// src/repositories/userRepository.js
// ... existing imports and code
async findByVerificationToken(token) {
return await User.findOne({ where: { verificationToken: token } });
}
async verifyUser(userId) {
const user = await User.findByPk(userId);
if (!user) return null;
await user.update({ isVerified: true, verificationToken: null });
return user;
}
module.exports = new UserRepository();
```
**b. **Password Reset Functionality**
Allow users to reset their passwords if they forget them.
**i. **Add Reset Token Fields to User Model
Add `resetPasswordToken` and `resetPasswordExpires` fields to the `User` model.
**`models/user.js`:**
```javascript
// models/user.js
// ... existing imports and code
User.init(
{
// ... existing fields
resetPasswordToken: {
type: DataTypes.STRING,
allowNull: true,
},
resetPasswordExpires: {
type: DataTypes.DATE,
allowNull: true,
},
},
{
sequelize,
modelName: 'User',
tableName: 'Users',
timestamps: true,
paranoid: true,
}
);
// ... existing associations and exports
```
**ii. **Create Migration for Reset Fields**
```bash
npx sequelize-cli migration:generate --name add-reset-fields-to-users
```
**`migrations/XXXXXX-add-reset-fields-to-users.js`:**
```javascript
'use strict';
module.exports = {
up: async (queryInterface, Sequelize) => {
await queryInterface.addColumn('Users', 'resetPasswordToken', {
type: Sequelize.STRING,
allowNull: true,
});
await queryInterface.addColumn('Users', 'resetPasswordExpires', {
type: Sequelize.DATE,
allowNull: true,
});
},
down: async (queryInterface, Sequelize) => {
await queryInterface.removeColumn('Users', 'resetPasswordExpires');
await queryInterface.removeColumn('Users', 'resetPasswordToken');
}
};
```
**Run Migration:**
```bash
npx sequelize-cli db:migrate
```
**iii. **Create Password Reset Endpoints**
**`src/routes/v1/userRoutes.js`:**
```javascript
// src/routes/v1/userRoutes.js
// ... existing imports and routes
const { requestPasswordReset, resetPassword } = require('../../controllers/userController');
// Request Password Reset
router.post(
'/request-password-reset',
[
body('email').isEmail().withMessage('Valid email is required'),
],
validate,
requestPasswordReset
);
// Reset Password
router.post(
'/reset-password',
[
body('token').notEmpty().withMessage('Reset token is required'),
body('newPassword').isLength({ min: 6 }).withMessage('Password must be at least 6 characters long'),
],
validate,
resetPassword
);
module.exports = router;
```
**`src/controllers/userController.js`:**
```javascript
// src/controllers/userController.js
const crypto = require('crypto');
const emailService = require('../services/emailService');
// Request Password Reset
exports.requestPasswordReset = async (req, res, next) => {
try {
const { email } = req.body;
// Find user by email
const user = await userService.findUserByEmail(email);
if (!user) {
throw new CustomError('User with this email does not exist', 404);
}
// Generate reset token
const resetToken = crypto.randomBytes(32).toString('hex');
const resetTokenHash = crypto.createHash('sha256').update(resetToken).digest('hex');
const resetTokenExpiry = Date.now() + 3600000; // 1 hour
// Update user with reset token
await userService.updateResetToken(user.id, resetTokenHash, resetTokenExpiry);
// Send password reset email
await emailService.sendPasswordResetEmail(user, resetToken);
res.status(200).json({
success: true,
message: 'Password reset email sent successfully',
});
} catch (error) {
next(error);
}
};
// Reset Password
exports.resetPassword = async (req, res, next) => {
try {
const { token, newPassword } = req.body;
// Hash the provided token
const hashedToken = crypto.createHash('sha256').update(token).digest('hex');
// Find user by reset token and check expiry
const user = await userService.findUserByResetToken(hashedToken, Date.now());
if (!user) {
throw new CustomError('Invalid or expired reset token', 400);
}
// Hash the new password
const hashedPassword = await bcrypt.hash(newPassword, 10);
// Update user's password and remove reset token
await userService.resetUserPassword(user.id, hashedPassword);
res.status(200).json({
success: true,
message: 'Password has been reset successfully',
});
} catch (error) {
next(error);
}
};
```
**`src/services/userService.js`:**
```javascript
// src/services/userService.js
// ... existing imports and code
const updateResetToken = async (userId, token, expires) => {
const user = await userRepository.findById(userId);
if (!user) return null;
await user.update({
resetPasswordToken: token,
resetPasswordExpires: expires,
});
return user;
};
const findUserByResetToken = async (token, currentTime) => {
return await userRepository.findByResetToken(token, currentTime);
};
const resetUserPassword = async (userId, newPassword) => {
const user = await userRepository.findById(userId);
if (!user) return null;
await user.update({
password: newPassword,
resetPasswordToken: null,
resetPasswordExpires: null,
});
return user;
};
module.exports = {
// ... existing exports
updateResetToken,
findUserByResetToken,
resetUserPassword,
};
```
**`src/repositories/userRepository.js`:**
```javascript
// src/repositories/userRepository.js
// ... existing imports and code
async findByResetToken(token, currentTime) {
return await User.findOne({
where: {
resetPasswordToken: token,
resetPasswordExpires: { [Op.gt]: currentTime },
},
});
}
async updateResetToken(userId, token, expires) {
const user = await User.findByPk(userId);
if (!user) return null;
await user.update({
resetPasswordToken: token,
resetPasswordExpires: expires,
});
return user;
}
async resetUserPassword(userId, newPassword) {
const user = await User.findByPk(userId);
if (!user) return null;
await user.update({
password: newPassword,
resetPasswordToken: null,
resetPasswordExpires: null,
});
return user;
}
module.exports = new UserRepository();
```
**iv. **Update Password Reset Email Template**
Ensure that the password reset email contains a secure link with the reset token.
**`src/services/emailService.js`:**
```javascript
// src/services/emailService.js
// ... existing code
const sendPasswordResetEmail = async (user, resetToken) => {
const resetUrl = `${process.env.FRONTEND_URL}/reset-password?token=${resetToken}`;
const mailOptions = {
from: `"EduConnect Support" <${process.env.EMAIL_USER}>`,
to: user.email,
subject: 'Reset Your EduConnect Password',
html: `
<p>Hello ${user.username},</p>
<p>You requested to reset your password. Click the link below to proceed:</p>
<a href="${resetUrl}">Reset Password</a>
<p>This link will expire in 1 hour.</p>
<p>If you did not request this, please ignore this email.</p>
`,
};
try {
await transporter.sendMail(mailOptions);
} catch (error) {
throw new CustomError('Failed to send password reset email', 500);
}
};
```
---
#### 5. **Securing Token Management with Refresh Tokens**
Proper management of JWT access and refresh tokens is essential for maintaining security and user experience.
**a. **Store Refresh Tokens Securely**
Rather than sending refresh tokens to the client and storing them in the database, use a secure storage mechanism like **Redis**. This allows for easy invalidation and scalability.
**b. **Update Token Generation to Include Refresh Token Storage**
**`src/controllers/userController.js`:**
```javascript
// src/controllers/userController.js
const { redisClient } = require('../../config/redis');
// ... existing code
// Generate and Store Refresh Token
const generateRefreshToken = (user) => {
return jwt.sign(
{ id: user.id },
process.env.JWT_REFRESH_SECRET,
{ expiresIn: '7d' }
);
};
// After generating refresh token
// Store refresh token in Redis with user ID as key
await redisClient.set(`refresh_token_${user.id}`, refreshToken, {
EX: 7 * 24 * 60 * 60, // 7 days in seconds
});
```
**c. **Modify Refresh Token Endpoint to Validate Against Redis**
Ensure that refresh tokens are valid and haven't been revoked.
**`src/controllers/userController.js`:**
```javascript
// src/controllers/userController.js
// Refresh Token
exports.refreshToken = async (req, res, next) => {
try {
const { refreshToken } = req.body;
if (!refreshToken) {
throw new CustomError('Refresh token is required', 400);
}
// Verify refresh token
const decoded = jwt.verify(refreshToken, process.env.JWT_REFRESH_SECRET);
const userId = decoded.id;
// Check if refresh token exists in Redis
const storedRefreshToken = await redisClient.get(`refresh_token_${userId}`);
if (storedRefreshToken !== refreshToken) {
throw new CustomError('Invalid refresh token', 401);
}
// Generate new access token
const user = await userService.findUserById(userId);
if (!user) {
throw new CustomError('User not found', 404);
}
const newToken = generateToken(user);
res.status(200).json({
success: true,
data: {
token: newToken,
},
});
} catch (error) {
if (error.name === 'TokenExpiredError') {
return next(new CustomError('Refresh token expired', 401));
}
next(new CustomError('Invalid refresh token', 401));
}
};
```
**d. **Invalidate Refresh Tokens on Logout**
Ensure that refresh tokens are invalidated when users log out.
**Add Logout Endpoint:**
**`src/routes/v1/userRoutes.js`:**
```javascript
// src/routes/v1/userRoutes.js
// ... existing imports and routes
const { logout } = require('../../controllers/userController');
// Logout User
router.post('/logout', authMiddleware, logout);
module.exports = router;
```
**`src/controllers/userController.js`:**
```javascript
// src/controllers/userController.js
// Logout User
exports.logout = async (req, res, next) => {
try {
const userId = req.user.id;
const refreshTokenKey = `refresh_token_${userId}`;
// Delete refresh token from Redis
await redisClient.del(refreshTokenKey);
res.status(200).json({
success: true,
message: 'Logged out successfully',
});
} catch (error) {
next(error);
}
};
```
---
#### 6. **Enhancing Authorization Middleware**
Ensure that authorization checks are thorough and cover all necessary scenarios.
**a. **Implement Ownership Checks**
For certain resources, users should only access or modify their own data.
**Example: Ensuring Users Can Only Access Their Own Profiles**
**`src/routes/v1/userRoutes.js`:**
```javascript
// src/routes/v1/userRoutes.js
// ... existing imports and routes
const { getOwnProfile } = require('../../controllers/userController');
// Get Own Profile
router.get('/me', authMiddleware, getOwnProfile);
module.exports = router;
```
**`src/controllers/userController.js`:**
```javascript
// src/controllers/userController.js
// Get Own Profile
exports.getOwnProfile = async (req, res, next) => {
try {
const user = await userService.findUserById(req.user.id);
if (!user) {
throw new CustomError('User not found', 404);
}
res.status(200).json({
success: true,
data: {
id: user.id,
username: user.username,
email: user.email,
roles: user.roles,
isVerified: user.isVerified,
createdAt: user.createdAt,
updatedAt: user.updatedAt,
},
});
} catch (error) {
next(error);
}
};
```
**b. **Restrict Access to Admin Routes**
Ensure that only admins can access sensitive routes.
**`src/routes/v1/adminRoutes.js`:**
```javascript
// src/routes/v1/adminRoutes.js
// ... existing code
// Apply 'admin' role authorization
router.use(authorize('admin'));
// Admin-specific routes
// For example, user management routes
module.exports = router;
```
**c. **Implement Resource-Based Authorization**
For resources like courses, ensure that only owners (e.g., instructors) or admins can modify them.
**Example: Ensuring Only Course Instructors or Admins Can Update/Delete Courses**
Already partially implemented in **Step 4**, but reinforce as needed.
**`src/controllers/courseController.js`:**
```javascript
// src/controllers/courseController.js
exports.updateCourse = async (req, res, next) => {
try {
const { courseId } = req.params;
const updates = req.body;
// Fetch course
const course = await courseService.getCourseById(courseId);
if (!course) {
throw new CustomError('Course not found', 404);
}
// Check ownership or admin role
if (req.user.roles.includes('admin') || course.instructorId === req.user.id) {
const updatedCourse = await courseService.updateCourse(courseId, updates);
res.status(200).json({
success: true,
data: updatedCourse,
});
} else {
throw new CustomError('Access denied: Not authorized to update this course', 403);
}
} catch (error) {
next(error);
}
};
```
---
#### 7. **Adhering to Security Best Practices**
Implement industry-standard security measures to protect user data and system integrity.
**a. **Secure Password Storage**
- **Hash Passwords:** Already implemented using bcrypt.
- **Use Strong Salt Rounds:** Use at least 10 salt rounds to hash passwords.
**b. **Use HTTPS**
- Ensure that all communications between clients and the server are encrypted using HTTPS.
- In development, use tools like **mkcert** to create local SSL certificates.
- In production, obtain certificates from a trusted Certificate Authority (e.g., Let's Encrypt).
**c. **Implement Content Security Policy (CSP)**
Use **Helmet.js** to set security-related HTTP headers.
**`src/app.js`:**
```javascript
// src/app.js
const helmet = require('helmet');
app.use(helmet());
// Customize CSP as needed
app.use(
helmet.contentSecurityPolicy({
directives: {
defaultSrc: ["'self'"],
scriptSrc: ["'self'", 'trusted.cdn.com'],
styleSrc: ["'self'", 'trusted.cdn.com'],
imgSrc: ["'self'", 'data:', 'trusted.cdn.com'],
connectSrc: ["'self'", 'api.educonnect.com', 'socket.io'],
// Add more directives as needed
},
})
);
```
**d. **Prevent Cross-Site Scripting (XSS) and SQL Injection**
- **Input Validation and Sanitization:** Already implemented using **express-validator**.
- **ORM Usage:** Using Sequelize and Mongoose helps prevent SQL injection as they handle parameterization.
**e. **Rate Limiting**
Already implemented in **Step 2** using **express-rate-limit** to prevent brute-force attacks.
**f. **Secure Cookies (If Using Sessions)**
If implementing sessions, ensure cookies are secure.
**Example:**
```javascript
// src/app.js
app.use(
session({
secret: process.env.SESSION_SECRET,
resave: false,
saveUninitialized: false,
cookie: {
secure: process.env.NODE_ENV === 'production', // Use HTTPS in production
httpOnly: true,
maxAge: 24 * 60 * 60 * 1000, // 1 day
},
})
);
```
**g. **Implement CORS Properly**
Already configured in **Step 2**. Ensure that only trusted origins can access the API.
**h. **Enable HTTP Strict Transport Security (HSTS)**
Enforce the use of HTTPS.
**`src/app.js`:**
```javascript
// src/app.js
app.use(
helmet.hsts({
maxAge: 31536000, // 1 year in seconds
includeSubDomains: true,
preload: true,
})
);
```
**i. **Limit Payload Size**
Prevent denial-of-service (DoS) attacks by limiting request payload sizes.
**`src/app.js`:**
```javascript
// src/app.js
app.use(express.json({ limit: '10kb' })); // Limit JSON payloads to 10KB
app.use(express.urlencoded({ extended: false, limit: '10kb' }));
```
---
#### 8. **Comprehensive Testing of Authentication and Authorization**
Ensure that authentication and authorization mechanisms work as intended through thorough testing.
**a. **Unit Tests for Authorization Middleware**
**`tests/unit/authorizeMiddleware.test.js`:**
```javascript
// tests/unit/authorizeMiddleware.test.js
const authorize = require('../../src/middleware/authorize');
const CustomError = require('../../src/utils/CustomError');
describe('Authorization Middleware', () => {
let req, res, next;
beforeEach(() => {
req = { user: { roles: ['student'] } };
res = {};
next = jest.fn();
});
it('should allow access if user has required role', () => {
const middleware = authorize(['student']);
middleware(req, res, next);
expect(next).toHaveBeenCalled();
});
it('should deny access if user lacks required role', () => {
const middleware = authorize(['admin']);
middleware(req, res, next);
expect(next).toHaveBeenCalledWith(expect.any(CustomError));
const error = next.mock.calls[0][0];
expect(error.message).toBe('Access denied: Insufficient permissions');
expect(error.status).toBe(403);
});
it('should allow access if user has one of the required roles', () => {
req.user.roles = ['instructor'];
const middleware = authorize(['admin', 'instructor']);
middleware(req, res, next);
expect(next).toHaveBeenCalled();
});
it('should handle single string role input', () => {
const middleware = authorize('student');
middleware(req, res, next);
expect(next).toHaveBeenCalled();
});
});
```
**b. **Integration Tests for Protected Routes**
**Example: Testing Protected Admin Route**
**`tests/integration/adminRoutes.test.js`:**
```javascript
// tests/integration/adminRoutes.test.js
const request = require('supertest');
const app = require('../../src/app');
const { sequelize, User } = require('../../src/models');
const bcrypt = require('bcrypt');
const jwt = require('jsonwebtoken');
let adminToken;
let userToken;
beforeAll(async () => {
await sequelize.sync({ force: true });
// Create admin user
const adminPassword = await bcrypt.hash('adminpass', 10);
const admin = await User.create({
username: 'adminuser',
email: 'admin@example.com',
password: adminPassword,
roles: ['admin'],
isVerified: true,
});
// Create regular user
const userPassword = await bcrypt.hash('userpass', 10);
const user = await User.create({
username: 'regularuser',
email: 'user@example.com',
password: userPassword,
roles: ['student'],
isVerified: true,
});
// Generate tokens
adminToken = jwt.sign(
{ id: admin.id, roles: admin.roles },
process.env.JWT_SECRET,
{ expiresIn: '1h' }
);
userToken = jwt.sign(
{ id: user.id, roles: user.roles },
process.env.JWT_SECRET,
{ expiresIn: '1h' }
);
});
afterAll(async () => {
await sequelize.close();
});
describe('Admin Routes', () => {
it('should allow admin to get all users', async () => {
const res = await request(app)
.get('/v1/admin/users')
.set('Authorization', `Bearer ${adminToken}`);
expect(res.statusCode).toEqual(200);
expect(res.body.success).toBe(true);
expect(res.body.data.length).toBe(2);
});
it('should deny access to regular users', async () => {
const res = await request(app)
.get('/v1/admin/users')
.set('Authorization', `Bearer ${userToken}`);
expect(res.statusCode).toEqual(403);
expect(res.body.success).toBe(false);
expect(res.body.message).toBe('Access denied: Insufficient permissions');
});
it('should allow admin to update user roles', async () => {
const user = await User.findOne({ where: { email: 'user@example.com' } });
const res = await request(app)
.put(`/v1/admin/users/${user.id}/roles`)
.set('Authorization', `Bearer ${adminToken}`)
.send({ roles: ['instructor'] });
expect(res.statusCode).toEqual(200);
expect(res.body.success).toBe(true);
expect(res.body.data.roles).toContain('instructor');
});
it('should allow admin to delete a user', async () => {
const user = await User.findOne({ where: { email: 'user@example.com' } });
const res = await request(app)
.delete(`/v1/admin/users/${user.id}`)
.set('Authorization', `Bearer ${adminToken}`);
expect(res.statusCode).toEqual(200);
expect(res.body.success).toBe(true);
expect(res.body.message).toBe('User deleted successfully');
});
});
```
**c. **Test OAuth Flows**
Testing OAuth flows can be complex due to external dependencies. Consider using tools like **nock** to mock external HTTP requests.
**Example: Mocking Google OAuth Callback**
**`tests/unit/googleOAuth.test.js`:**
```javascript
// tests/unit/googleOAuth.test.js
const passport = require('../../src/config/passport');
const request = require('supertest');
const app = require('../../src/app');
const nock = require('nock');
const { sequelize, User } = require('../../src/models');
const bcrypt = require('bcrypt');
beforeAll(async () => {
await sequelize.sync({ force: true });
});
afterAll(async () => {
await sequelize.close();
});
describe('Google OAuth', () => {
it('should register and login user via Google OAuth', async () => {
// Mock Google OAuth response
nock('https://accounts.google.com')
.post('/o/oauth2/token')
.reply(200, {
access_token: 'fake_access_token',
id_token: 'fake_id_token',
});
nock('https://www.googleapis.com')
.get('/oauth2/v3/userinfo')
.reply(200, {
sub: '1234567890',
email: 'googleuser@example.com',
email_verified: true,
name: 'Google User',
picture: 'https://example.com/avatar.jpg',
});
// Simulate OAuth callback
const res = await request(app)
.get('/v1/auth/google/callback?code=valid_code')
.expect(200);
expect(res.body.success).toBe(true);
expect(res.body.data).toHaveProperty('token');
expect(res.body.data).toHaveProperty('refreshToken');
// Verify user in database
const user = await User.findOne({ where: { email: 'googleuser@example.com' } });
expect(user).not.toBeNull();
expect(user.username).toBe('Google User');
expect(user.roles).toContain('student');
expect(user.isVerified).toBe(true);
});
});
```
**Note:** OAuth testing often requires integration testing environments or using service-specific testing tools. The above example uses **nock** to mock external requests but may need adjustments based on the OAuth provider's specifics.
---
#### 9. **Implement Multi-Factor Authentication (Optional)**
For enhanced security, consider adding Multi-Factor Authentication (MFA). This can be achieved using Time-based One-Time Passwords (TOTP) with apps like Google Authenticator.
**a. **Install Necessary Packages**
```bash
npm install speakeasy qrcode
```
**b. **Add MFA Fields to User Model**
Add `mfaEnabled` and `mfaSecret` fields to the `User` model.
**`models/user.js`:**
```javascript
// models/user.js
// ... existing imports and code
User.init(
{
// ... existing fields
mfaEnabled: {
type: DataTypes.BOOLEAN,
defaultValue: false,
},
mfaSecret: {
type: DataTypes.STRING,
allowNull: true,
},
},
{
sequelize,
modelName: 'User',
tableName: 'Users',
timestamps: true,
paranoid: true,
}
);
// ... existing associations and exports
```
**c. **Create Migration for MFA Fields**
```bash
npx sequelize-cli migration:generate --name add-mfa-fields-to-users
```
**`migrations/XXXXXX-add-mfa-fields-to-users.js`:**
```javascript
'use strict';
module.exports = {
up: async (queryInterface, Sequelize) => {
await queryInterface.addColumn('Users', 'mfaEnabled', {
type: Sequelize.BOOLEAN,
defaultValue: false,
});
await queryInterface.addColumn('Users', 'mfaSecret', {
type: Sequelize.STRING,
allowNull: true,
});
},
down: async (queryInterface, Sequelize) => {
await queryInterface.removeColumn('Users', 'mfaSecret');
await queryInterface.removeColumn('Users', 'mfaEnabled');
}
};
```
**Run Migration:**
```bash
npx sequelize-cli db:migrate
```
**d. **MFA Setup Controller**
Allow users to enable MFA by generating a secret and QR code.
**`src/controllers/mfaController.js`:**
```javascript
// src/controllers/mfaController.js
const speakeasy = require('speakeasy');
const qrcode = require('qrcode');
const userService = require('../services/userService');
const CustomError = require('../utils/CustomError');
// Enable MFA - Generate Secret and QR Code
exports.enableMFA = async (req, res, next) => {
try {
const userId = req.user.id;
// Generate TOTP secret
const secret = speakeasy.generateSecret({
name: `EduConnect (${req.user.email})`,
});
// Save secret to user
await userService.updateMFASecret(userId, secret.base32);
// Generate QR code
const qrCodeDataURL = await qrcode.toDataURL(secret.otpauth_url);
res.status(200).json({
success: true,
data: {
secret: secret.base32,
qrCode: qrCodeDataURL,
},
message: 'Scan the QR code with your authenticator app',
});
} catch (error) {
next(error);
}
};
// Verify MFA Token and Enable MFA
exports.verifyMFA = async (req, res, next) => {
try {
const userId = req.user.id;
const { token } = req.body;
const user = await userService.findUserById(userId);
if (!user || !user.mfaSecret) {
throw new CustomError('MFA not set up', 400);
}
const verified = speakeasy.totp.verify({
secret: user.mfaSecret,
encoding: 'base32',
token,
});
if (!verified) {
throw new CustomError('Invalid MFA token', 400);
}
// Enable MFA
await userService.enableMFA(userId);
res.status(200).json({
success: true,
message: 'MFA enabled successfully',
});
} catch (error) {
next(error);
}
};
// Disable MFA
exports.disableMFA = async (req, res, next) => {
try {
const userId = req.user.id;
const { token } = req.body;
const user = await userService.findUserById(userId);
if (!user || !user.mfaSecret || !user.mfaEnabled) {
throw new CustomError('MFA not enabled', 400);
}
const verified = speakeasy.totp.verify({
secret: user.mfaSecret,
encoding: 'base32',
token,
});
if (!verified) {
throw new CustomError('Invalid MFA token', 400);
}
// Disable MFA
await userService.disableMFA(userId);
res.status(200).json({
success: true,
message: 'MFA disabled successfully',
});
} catch (error) {
next(error);
}
};
```
**e. **MFA Service**
**`src/services/userService.js`:**
```javascript
// src/services/userService.js
// ... existing imports and code
const updateMFASecret = async (userId, secret) => {
const user = await userRepository.findById(userId);
if (!user) return null;
await user.update({ mfaSecret: secret });
return user;
};
const enableMFA = async (userId) => {
const user = await userRepository.findById(userId);
if (!user) return null;
await user.update({ mfaEnabled: true });
return user;
};
const disableMFA = async (userId) => {
const user = await userRepository.findById(userId);
if (!user) return null;
await user.update({ mfaEnabled: false, mfaSecret: null });
return user;
};
module.exports = {
// ... existing exports
updateMFASecret,
enableMFA,
disableMFA,
};
```
**f. **MFA Repository**
**`src/repositories/userRepository.js`:**
```javascript
// src/repositories/userRepository.js
// ... existing imports and code
module.exports = new UserRepository();
```
**g. **Add MFA Verification During Login**
Modify the login process to require MFA if enabled.
**`src/controllers/userController.js`:**
```javascript
// src/controllers/userController.js
// Modify Login User
exports.login = async (req, res, next) => {
try {
const { email, password, mfaToken } = req.body;
// Find user
const user = await userService.findUserByEmail(email);
if (!user) {
throw new CustomError('Invalid email or password', 401);
}
// Check if user is verified
if (!user.isVerified) {
throw new CustomError('Please verify your email before logging in', 403);
}
// Compare password
const isMatch = await bcrypt.compare(password, user.password);
if (!isMatch) {
throw new CustomError('Invalid email or password', 401);
}
// If MFA is enabled, verify token
if (user.mfaEnabled) {
if (!mfaToken) {
throw new CustomError('MFA token is required', 401);
}
const verified = speakeasy.totp.verify({
secret: user.mfaSecret,
encoding: 'base32',
token: mfaToken,
});
if (!verified) {
throw new CustomError('Invalid MFA token', 401);
}
}
// Generate tokens
const token = generateToken(user);
const refreshToken = generateRefreshToken(user);
// Store refresh token in Redis
await redisClient.set(`refresh_token_${user.id}`, refreshToken, {
EX: 7 * 24 * 60 * 60, // 7 days
});
res.status(200).json({
success: true,
data: {
token,
refreshToken,
},
});
} catch (error) {
next(error);
}
};
```
**h. **Update Login Validation**
Ensure that the login endpoint can handle optional MFA tokens.
**`src/routes/v1/userRoutes.js`:**
```javascript
// src/routes/v1/userRoutes.js
// ... existing imports and routes
// Login User with optional MFA token
router.post(
'/login',
[
body('email').isEmail().withMessage('Valid email is required'),
body('password').notEmpty().withMessage('Password is required'),
body('mfaToken').optional().isNumeric().withMessage('MFA token must be numeric'),
],
validate,
login
);
// ... existing code
```
---
#### 10. **Summary**
In **Step 5: Authentication and Authorization**, we've significantly strengthened the security and control mechanisms of the **EduConnect** backend by:
- **Implementing Robust RBAC:** Defined clear roles and enforced permissions through middleware.
- **Integrating OAuth2:** Enabled users to log in via Google, Facebook, and GitHub using Passport.js strategies.
- **Adding Account Verification:** Ensured users verify their email addresses before accessing the system.
- **Implementing Password Reset:** Allowed users to securely reset their passwords through email links.
- **Securing Token Management:** Stored refresh tokens securely in Redis and enforced token validation.
- **Enhancing Authorization Middleware:** Implemented ownership and role-based checks for resource access.
- **Adhering to Security Best Practices:** Applied industry-standard security measures, including secure password storage, HTTPS enforcement, input validation, and rate limiting.
- **Conducting Comprehensive Testing:** Ensured all authentication and authorization flows work correctly through unit and integration tests.
- **Optional MFA Integration:** Provided the groundwork for adding Multi-Factor Authentication to bolster security further.
---
### **Next Steps**
With authentication and authorization robustly implemented, the next phase is **Step 6: Implement Adaptive Learning Algorithms**. In this step, we'll develop and integrate algorithms that tailor the learning experience based on user performance and progress, enhancing the personalized learning journey for each user.
Feel free to ask any questions or request further assistance regarding **Step 5: Authentication and Authorization**!
===
### Step 6: Implement Adaptive Learning Algorithms
In **Step 6: Implement Adaptive Learning Algorithms**, we'll develop and integrate algorithms that tailor the learning experience based on each user's performance and progress. Adaptive learning enhances personalization, ensuring that users receive content that matches their proficiency levels, learning speeds, and interests. This step involves:
1. **Defining Adaptive Learning Objectives and Metrics**
2. **Designing the Adaptive Learning Architecture**
3. **Implementing Data Collection and Analysis**
4. **Developing Adaptive Algorithms**
5. **Integrating Adaptive Learning with Existing Services**
6. **Notifying Users of Adaptations**
7. **Monitoring and Improving Algorithms**
8. **Ensuring Privacy and Ethical Considerations**
9. **Comprehensive Testing of Adaptive Features**
Let's delve into each of these components in detail.
---
#### 1. **Defining Adaptive Learning Objectives and Metrics**
**Objectives:**
- **Personalization:** Adjust course content based on individual user performance and preferences.
- **Efficiency:** Optimize learning paths to reduce redundancy and focus on areas needing improvement.
- **Engagement:** Enhance user engagement by presenting challenging yet achievable material.
**Metrics to Track:**
- **Quiz Scores:** Assess comprehension and retention.
- **Completion Rates:** Monitor module and course completion.
- **Time Spent:** Evaluate user engagement and pacing.
- **Interaction Patterns:** Analyze how users interact with materials (e.g., video views, document reads).
- **Feedback and Ratings:** Incorporate user feedback to adjust content quality.
---
#### 2. **Designing the Adaptive Learning Architecture**
**Components:**
1. **Data Layer:**
- **User Progress Data:** Stored in MongoDB (as per previous steps).
- **Course Content Data:** Managed via PostgreSQL.
- **Adaptive Parameters:** Define thresholds and rules for adaptation.
2. **Processing Layer:**
- **Analytics Service:** Processes collected data to identify patterns.
- **Adaptation Engine:** Applies algorithms to adjust learning paths.
3. **Integration Layer:**
- **API Endpoints:** Expose adaptive recommendations to the frontend.
- **Notification Service:** Communicates changes or recommendations to users.
**Data Flow:**
1. **Data Collection:** Gather user interactions, quiz results, and feedback.
2. **Data Analysis:** Process and analyze data to assess user performance.
3. **Decision Making:** Determine necessary adaptations based on analysis.
4. **Content Adjustment:** Modify learning paths, suggest materials, or adjust difficulty.
5. **User Notification:** Inform users about changes and recommendations.
---
#### 3. **Implementing Data Collection and Analysis**
Ensure comprehensive data collection to feed into adaptive algorithms.
**a. **Enhance Progress Tracking**
Expand the existing progress tracking to capture more detailed metrics.
**`models/mongoose/UserProgress.js`:**
```javascript
// models/mongoose/UserProgress.js
const mongoose = require('mongoose');
const UserProgressSchema = new mongoose.Schema(
{
userId: {
type: String,
required: true,
index: true,
},
courseId: {
type: String,
required: true,
index: true,
},
completedModules: {
type: [String], // Array of module IDs or names
default: [],
},
quizScores: {
type: Map,
of: Number, // Map of quizId to score
default: {},
},
timeSpent: {
type: Map,
of: Number, // Map of moduleId to time in minutes
default: {},
},
interactionLogs: {
type: Array, // Detailed interaction logs
default: [],
},
feedback: {
type: Map,
of: String, // Map of moduleId or quizId to user feedback
default: {},
},
adaptiveData: {
type: Map,
of: mongoose.Schema.Types.Mixed, // Data used by adaptive algorithms
default: {},
},
},
{ timestamps: true }
);
module.exports = mongoose.model('UserProgress', UserProgressSchema);
```
**b. **Log User Interactions**
Capture detailed interaction data to inform adaptations.
**`src/services/progressService.js`:**
```javascript
// src/services/progressService.js
const userProgressRepository = require('../repositories/userProgressRepository');
const courseService = require('./courseService');
const CustomError = require('../utils/CustomError');
const getUserProgress = async (userId) => {
return await userProgressRepository.findByUserId(userId);
};
const updateUserProgress = async (userId, { courseId, completedModules, quizScores, timeSpent, interactions, feedback }) => {
// Validate course existence
const course = await courseService.getCourseById(courseId);
if (!course) {
throw new CustomError('Course not found', 404);
}
// Update progress
const progress = await userProgressRepository.updateProgress(userId, courseId, {
completedModules,
quizScores,
timeSpent,
interactions,
feedback
});
return progress;
};
module.exports = {
getUserProgress,
updateUserProgress,
};
```
**c. **Implement Interaction Logging**
Track various user interactions for analysis.
**`src/controllers/progressController.js`:**
```javascript
// src/controllers/progressController.js
exports.updateUserProgress = async (req, res, next) => {
try {
const userId = req.user.id;
const { courseId, completedModules, quizScores, timeSpent, interactions, feedback } = req.body;
const updatedProgress = await progressService.updateUserProgress(userId, {
courseId,
completedModules,
quizScores,
timeSpent,
interactions,
feedback
});
res.status(200).json({
success: true,
data: updatedProgress,
});
} catch (error) {
next(error);
}
};
```
**Example Interaction Log Entry:**
```json
{
"action": "view_video",
"moduleId": "module-uuid",
"videoId": "video-uuid",
"timestamp": "2024-04-01T12:00:00Z",
"duration": 15 // in minutes
}
```
---
#### 4. **Developing Adaptive Algorithms**
Implement algorithms that analyze user data and make informed decisions to adjust the learning experience.
**a. **Rule-Based Adaptation**
Simple and interpretable, based on predefined rules.
**Example Rules:**
- **If quiz score < 70%**, recommend revisiting the corresponding module.
- **If user spends > 60 minutes on a module without completing**, suggest interactive materials.
- **If completion rate < 50% for a course**, send motivational notifications.
**b. **Machine Learning-Based Adaptation**
Advanced approach using machine learning to predict user needs and preferences.
**Note:** Implementing ML models requires data preparation, model training, and integration. For simplicity, we'll focus on a rule-based approach.
**c. **Implementing a Rule-Based Adaptation Engine**
**`src/services/adaptiveLearningService.js`:**
```javascript
// src/services/adaptiveLearningService.js
const userProgressService = require('./progressService');
const learningMaterialService = require('./learningMaterialService');
const courseService = require('./courseService');
const { io } = require('../index'); // Import the Socket.io instance
const CustomError = require('../utils/CustomError');
const adjustLearningPath = async (userId, courseId) => {
// Fetch user progress
const progress = await userProgressService.getUserProgress(userId);
const courseProgress = progress.find(p => p.courseId === courseId);
if (!courseProgress) {
// Initialize progress if not exists
await userProgressService.updateUserProgress(userId, {
courseId,
completedModules: [],
quizScores: {},
timeSpent: {},
interactions: [],
feedback: {}
});
return;
}
const recommendations = [];
// Rule 1: Recommend revisiting modules with low quiz scores
const lowScoreQuizzes = [];
for (const [quizId, score] of courseProgress.quizScores.entries()) {
if (score < 70) {
lowScoreQuizzes.push(quizId);
// Find corresponding module
const moduleId = await getModuleIdByQuizId(courseId, quizId);
if (moduleId) {
recommendations.push({
type: 'revisit_module',
moduleId,
message: `Consider revisiting Module ${moduleId} to improve your understanding.`,
});
}
}
}
// Rule 2: Suggest additional materials for modules where user spends excessive time
for (const [moduleId, duration] of courseProgress.timeSpent.entries()) {
if (duration > 60 && !courseProgress.completedModules.includes(moduleId)) {
recommendations.push({
type: 'additional_material',
moduleId,
message: `You're spending a lot of time on Module ${moduleId}. Here are some additional resources to help you.`,
});
}
}
// Rule 3: Encourage course completion if progress is low
const totalModules = await courseService.getTotalModules(courseId);
const completedModules = courseProgress.completedModules.length;
const completionRate = (completedModules / totalModules) * 100;
if (completionRate < 50) {
recommendations.push({
type: 'motivation',
message: 'Keep going! Completing more modules will enhance your learning experience.',
});
}
// Fetch recommended materials
for (const recommendation of recommendations) {
if (recommendation.type === 'revisit_module' || recommendation.type === 'additional_material') {
const materials = await learningMaterialService.getRecommendedMaterials(courseId, recommendation.moduleId);
recommendation.materials = materials;
}
}
// Update adaptive data if needed
// For example, store recommendations
await userProgressService.updateUserProgress(userId, {
courseId,
adaptiveData: { recommendations }
});
// Notify user via WebSocket
if (recommendations.length > 0) {
io.to(`user_${userId}`).emit('ADAPTIVE_RECOMMENDATIONS', {
courseId,
recommendations,
});
}
};
// Helper function to get module ID by quiz ID
const getModuleIdByQuizId = async (courseId, quizId) => {
// Implement logic to map quizId to moduleId
// This could involve querying the database or maintaining a mapping
// Placeholder implementation:
const course = await courseService.getCourseById(courseId);
if (!course) return null;
for (const module of course.modules) {
if (module.quizId === quizId) {
return module.id;
}
}
return null;
};
module.exports = {
adjustLearningPath,
};
```
**d. **Define Recommended Materials Retrieval**
Fetch materials based on recommendations.
**`src/services/learningMaterialService.js`:**
```javascript
// src/services/learningMaterialService.js
const LearningMaterial = require('../models/mongoose/LearningMaterial');
const createLearningMaterial = async (materialData) => {
const material = new LearningMaterial(materialData);
return await material.save();
};
const getMaterialsByCourseId = async (courseId) => {
return await LearningMaterial.find({ courseId });
};
const getRecommendedMaterials = async (courseId, moduleId) => {
// Implement logic to fetch materials related to the specific module
// For example, filter by topic or difficulty level
// Placeholder implementation:
return await LearningMaterial.find({ courseId, relatedModule: moduleId });
};
module.exports = {
createLearningMaterial,
getMaterialsByCourseId,
getRecommendedMaterials,
};
```
**e. **Integrate Adaptation Engine with Quiz Submission**
Trigger the adaptation process after a user submits a quiz.
**`src/controllers/quizController.js`:**
```javascript
// src/controllers/quizController.js
const quizService = require('../services/quizService');
const adaptiveLearningService = require('../services/adaptiveLearningService');
const CustomError = require('../utils/CustomError');
exports.submitQuiz = async (req, res, next) => {
try {
const { quizId } = req.params;
const { answers } = req.body;
const userId = req.user.id;
// Submit quiz and get results
const quizResult = await quizService.submitQuiz(userId, quizId, answers);
// Adjust learning path based on results
await adaptiveLearningService.adjustLearningPath(userId, quizResult.courseId);
res.status(200).json({
success: true,
data: quizResult,
});
} catch (error) {
next(error);
}
};
```
---
#### 5. **Integrating Adaptive Learning with Existing Services**
Ensure seamless integration of adaptive learning functionalities with existing services like user progress tracking and course management.
**a. **Updating Course Service to Include Modules**
Define modules within courses to map quizzes and materials.
**`models/course.js`:**
```javascript
// models/course.js
'use strict';
const { Model } = require('sequelize');
module.exports = (sequelize, DataTypes) => {
class Course extends Model {
static associate(models) {
// Define associations here
Course.belongsTo(models.User, { foreignKey: 'instructorId', as: 'instructor' });
Course.hasMany(models.Quiz, { foreignKey: 'courseId', as: 'quizzes' });
// Add association for modules if separate
}
}
Course.init(
{
title: {
type: DataTypes.STRING,
allowNull: false
},
description: {
type: DataTypes.TEXT,
allowNull: false
},
category: {
type: DataTypes.STRING,
allowNull: false
},
instructorId: {
type: DataTypes.UUID,
allowNull: false,
references: {
model: 'Users',
key: 'id'
}
},
modules: {
type: DataTypes.JSONB, // Array of modules with details
allowNull: true,
},
// ... existing fields
},
{
sequelize,
modelName: 'Course',
tableName: 'Courses',
timestamps: true,
}
);
return Course;
};
```
**b. **Updating Course Creation to Include Modules**
Allow instructors to define modules during course creation.
**`src/controllers/courseController.js`:**
```javascript
// src/controllers/courseController.js
exports.createCourse = async (req, res, next) => {
try {
const { title, description, category, videos, materials, modules } = req.body;
const instructorId = req.user.id;
const course = await courseService.createCourse({
title,
description,
category,
videos,
materials,
modules,
instructorId,
});
res.status(201).json({
success: true,
data: course,
});
} catch (error) {
next(error);
}
};
```
**`src/services/courseService.js`:**
```javascript
// src/services/courseService.js
const courseRepository = require('../repositories/courseRepository');
const learningMaterialService = require('./learningMaterialService');
const quizService = require('./quizService');
const createCourse = async (courseData) => {
// Create course
const course = await courseRepository.create(courseData);
// Optionally, create modules, learning materials, and quizzes
if (courseData.modules && Array.isArray(courseData.modules)) {
for (const module of courseData.modules) {
// Create learning materials
if (module.materials && Array.isArray(module.materials)) {
for (const material of module.materials) {
await learningMaterialService.createLearningMaterial({
title: material.title,
content: material.content,
courseId: course.id,
resourceType: material.resourceType,
relatedModule: module.id,
});
}
}
// Create quizzes
if (module.quizzes && Array.isArray(module.quizzes)) {
for (const quiz of module.quizzes) {
const createdQuiz = await quizService.createQuiz({
courseId: course.id,
questions: quiz.questions,
timeLimit: quiz.timeLimit,
instructorId: course.instructorId,
moduleId: module.id, // Associate quiz with module
});
// Optionally, link quiz to module
// Update course with quiz IDs if necessary
}
}
}
}
return course;
};
// ... other methods
module.exports = {
createCourse,
// ... other exports
};
```
**c. **Extending Quiz Repository and Service to Include Module Associations**
**`src/repositories/quizRepository.js`:**
```javascript
// src/repositories/quizRepository.js
const { Quiz } = require('../models');
class QuizRepository {
async createQuiz({ courseId, questions, timeLimit, instructorId, moduleId }) {
return await Quiz.create({
courseId,
questions,
timeLimit,
instructorId,
moduleId
});
}
async findById(quizId) {
return await Quiz.findByPk(quizId, {
include: [{ model: require('./courseRepository').Course, as: 'course' }],
});
}
// ... other methods
}
module.exports = new QuizRepository();
```
**`models/quiz.js`:**
```javascript
// models/quiz.js
'use strict';
const { Model } = require('sequelize');
module.exports = (sequelize, DataTypes) => {
class Quiz extends Model {
static associate(models) {
// Define associations here
Quiz.belongsTo(models.Course, { foreignKey: 'courseId', as: 'course' });
// Optionally, associate with Module if modules are separate
}
}
Quiz.init(
{
title: {
type: DataTypes.STRING,
allowNull: false
},
courseId: {
type: DataTypes.UUID,
allowNull: false,
references: {
model: 'Courses',
key: 'id'
}
},
moduleId: { // Associate quiz with a module
type: DataTypes.STRING,
allowNull: true,
},
questions: {
type: DataTypes.JSONB,
allowNull: false
},
timeLimit: { // Time limit in minutes
type: DataTypes.INTEGER,
allowNull: true
}
},
{
sequelize,
modelName: 'Quiz',
tableName: 'Quizzes',
timestamps: true
}
);
return Quiz;
};
```
---
#### 6. **Notifying Users of Adaptations**
Communicate recommendations and adjustments to users in real-time using WebSockets or via email notifications.
**a. **Emit Recommendations via WebSockets**
Already partially implemented in the adaptation engine. Ensure frontend listens to these events.
**`src/services/adaptiveLearningService.js`:**
```javascript
// src/services/adaptiveLearningService.js
// ... existing code
// Notify user via WebSocket
if (recommendations.length > 0) {
io.to(`user_${userId}`).emit('ADAPTIVE_RECOMMENDATIONS', {
courseId,
recommendations,
});
}
```
**b. **Implement Email Notifications for Important Recommendations**
Optionally, send email summaries or critical recommendations.
**`src/services/adaptiveLearningService.js`:**
```javascript
// src/services/adaptiveLearningService.js
const emailService = require('./emailService');
const adjustLearningPath = async (userId, courseId) => {
// ... existing adaptation logic
if (recommendations.length > 0) {
// Fetch user email
const user = await userService.findUserById(userId);
if (user) {
// Compile recommendation messages
const messages = recommendations.map(rec => rec.message).join('\n');
// Send email
await emailService.sendAdaptiveRecommendations(user, messages);
}
// Emit via WebSocket
io.to(`user_${userId}`).emit('ADAPTIVE_RECOMMENDATIONS', {
courseId,
recommendations,
});
}
};
```
**`src/services/emailService.js`:**
```javascript
// src/services/emailService.js
// ... existing code
const sendAdaptiveRecommendations = async (user, messages) => {
const mailOptions = {
from: `"EduConnect Support" <${process.env.EMAIL_USER}>`,
to: user.email,
subject: 'Personalized Learning Recommendations',
html: `
<p>Hello ${user.username},</p>
<p>Based on your recent activity, we have some recommendations to enhance your learning experience:</p>
<p>${messages.replace(/\n/g, '<br>')}</p>
<p>Keep up the great work!</p>
<p>Best regards,<br>EduConnect Team</p>
`,
};
try {
await transporter.sendMail(mailOptions);
} catch (error) {
throw new CustomError('Failed to send adaptive recommendations email', 500);
}
};
```
---
#### 7. **Monitoring and Improving Algorithms**
Continuously evaluate the effectiveness of adaptive algorithms and refine them based on user feedback and performance data.
**a. **Implement Logging and Analytics**
Use logging tools to track algorithm performance and user responses.
**`src/services/adaptiveLearningService.js`:**
```javascript
// src/services/adaptiveLearningService.js
const logger = require('../utils/logger');
const adjustLearningPath = async (userId, courseId) => {
try {
// ... existing adaptation logic
logger.info(`Adaptive recommendations generated for user ${userId} in course ${courseId}`, { recommendations });
} catch (error) {
logger.error(`Error in adjustLearningPath for user ${userId}:`, error);
throw error;
}
};
```
**b. **Collect User Feedback**
Allow users to provide feedback on recommendations to assess their relevance and usefulness.
**`src/controllers/adaptiveLearningController.js`:**
```javascript
// src/controllers/adaptiveLearningController.js
const adaptiveLearningService = require('../services/adaptiveLearningService');
exports.submitFeedback = async (req, res, next) => {
try {
const userId = req.user.id;
const { courseId, recommendationId, feedback } = req.body;
// Update feedback in user progress
await adaptiveLearningService.submitFeedback(userId, courseId, recommendationId, feedback);
res.status(200).json({
success: true,
message: 'Feedback submitted successfully',
});
} catch (error) {
next(error);
}
};
```
**`src/services/adaptiveLearningService.js`:**
```javascript
// src/services/adaptiveLearningService.js
const submitFeedback = async (userId, courseId, recommendationId, feedback) => {
// Implement logic to store feedback
// For example, update adaptiveData in user progress
const progress = await userProgressService.getUserProgress(userId);
const courseProgress = progress.find(p => p.courseId === courseId);
if (courseProgress && courseProgress.adaptiveData && courseProgress.adaptiveData.recommendations) {
const recommendation = courseProgress.adaptiveData.recommendations.find(rec => rec.id === recommendationId);
if (recommendation) {
recommendation.feedback = feedback;
await userProgressService.updateUserProgress(userId, {
courseId,
adaptiveData: { recommendations: courseProgress.adaptiveData.recommendations }
});
}
}
};
```
**c. **Analyze Feedback for Algorithm Refinement**
Periodically analyze feedback and performance data to identify areas for improvement in the adaptation logic.
---
#### 8. **Ensuring Privacy and Ethical Considerations**
Handle user data responsibly to maintain privacy and adhere to ethical standards.
**a. **Data Minimization**
Collect only the data necessary for adapting learning paths.
**b. **User Consent**
Inform users about data collection practices and obtain consent where required.
**c. **Data Security**
Ensure that all collected data is securely stored and transmitted.
**d. **Transparent Adaptations**
Provide users with insights into how their data influences their learning experience.
**e. **Compliance with Regulations**
Adhere to data protection regulations such as GDPR or CCPA.
---
#### 9. **Comprehensive Testing of Adaptive Features**
Ensure that adaptive algorithms function correctly and effectively enhance the learning experience.
**a. **Unit Tests for Adaptive Services**
**`tests/unit/adaptiveLearningService.test.js`:**
```javascript
// tests/unit/adaptiveLearningService.test.js
const adaptiveLearningService = require('../../src/services/adaptiveLearningService');
const userProgressService = require('../../src/services/progressService');
const learningMaterialService = require('../../src/services/learningMaterialService');
const courseService = require('../../src/services/courseService');
const { io } = require('../../src/index');
const CustomError = require('../../src/utils/CustomError');
jest.mock('../../src/services/progressService');
jest.mock('../../src/services/learningMaterialService');
jest.mock('../../src/services/courseService');
jest.mock('../../src/index', () => ({
io: {
to: jest.fn().mockReturnThis(),
emit: jest.fn(),
},
}));
describe('Adaptive Learning Service', () => {
beforeEach(() => {
jest.clearAllMocks();
});
it('should recommend revisiting modules with low quiz scores', async () => {
const userId = 'user-uuid';
const courseId = 'course-uuid';
const progressData = [
{
courseId,
quizScores: new Map([
['quiz1', 65],
['quiz2', 80],
]),
completedModules: ['module1'],
timeSpent: new Map([
['module1', 30],
['module2', 45],
]),
},
];
userProgressService.getUserProgress.mockResolvedValue(progressData);
courseService.getCourseById.mockResolvedValue({
id: courseId,
modules: [
{ id: 'module1', quizId: 'quiz1' },
{ id: 'module2', quizId: 'quiz2' },
],
getTotalModules: () => 2,
});
learningMaterialService.getRecommendedMaterials.mockResolvedValue([
{ id: 'material1', title: 'Supplementary Material 1', url: 'https://...' },
]);
await adaptiveLearningService.adjustLearningPath(userId, courseId);
expect(userProgressService.getUserProgress).toHaveBeenCalledWith(userId);
expect(courseService.getCourseById).toHaveBeenCalledWith(courseId);
expect(learningMaterialService.getRecommendedMaterials).toHaveBeenCalledWith(courseId, 'module1');
expect(io.to).toHaveBeenCalledWith(`user_${userId}`);
expect(io.to().emit).toHaveBeenCalledWith('ADAPTIVE_RECOMMENDATIONS', expect.any(Object));
});
it('should initialize progress if not exists', async () => {
const userId = 'user-uuid';
const courseId = 'course-uuid';
const progressData = [];
userProgressService.getUserProgress.mockResolvedValue(progressData);
userProgressService.updateUserProgress.mockResolvedValue({});
await adaptiveLearningService.adjustLearningPath(userId, courseId);
expect(userProgressService.updateUserProgress).toHaveBeenCalledWith(userId, {
courseId,
completedModules: [],
quizScores: {},
timeSpent: {},
interactions: [],
feedback: {},
});
expect(io.to).not.toHaveBeenCalled();
});
// Add more tests for other rules and scenarios
});
```
**b. **Integration Tests for Adaptive Learning Flows**
Simulate user interactions and verify that adaptive recommendations are generated and communicated correctly.
**`tests/integration/adaptiveLearningFlow.test.js`:**
```javascript
// tests/integration/adaptiveLearningFlow.test.js
const request = require('supertest');
const app = require('../../src/app');
const { sequelize, User } = require('../../src/models');
const bcrypt = require('bcrypt');
const jwt = require('jsonwebtoken');
const userProgressService = require('../../src/services/progressService');
const adaptiveLearningService = require('../../src/services/adaptiveLearningService');
jest.mock('../../src/services/adaptiveLearningService');
beforeAll(async () => {
await sequelize.sync({ force: true });
// Create a user
const password = await bcrypt.hash('password123', 10);
const user = await User.create({
username: 'testuser',
email: 'testuser@example.com',
password,
roles: ['student'],
isVerified: true,
});
// Generate JWT
global.token = jwt.sign(
{ id: user.id, roles: user.roles },
process.env.JWT_SECRET,
{ expiresIn: '1h' }
);
});
afterAll(async () => {
await sequelize.close();
});
describe('Adaptive Learning Flow', () => {
it('should trigger adaptive learning after quiz submission', async () => {
// Mock user progress and course data
userProgressService.getUserProgress = jest.fn().mockResolvedValue([
{
courseId: 'course-uuid',
quizScores: new Map([
['quiz1', 65],
['quiz2', 80],
]),
completedModules: ['module1'],
timeSpent: new Map([
['module1', 30],
['module2', 45],
]),
},
]);
adaptiveLearningService.adjustLearningPath.mockResolvedValue();
// Submit a quiz
const res = await request(app)
.post('/v1/quizzes/quiz-uuid/submit')
.set('Authorization', `Bearer ${global.token}`)
.send({
answers: {
question1: 'optionA',
question2: 'optionB',
},
});
expect(res.statusCode).toEqual(200);
expect(res.body.success).toBe(true);
expect(adaptiveLearningService.adjustLearningPath).toHaveBeenCalledWith('user-uuid', 'course-uuid');
});
// Add more integration tests as needed
});
```
**c. **End-to-End (E2E) Testing**
Simulate real user journeys to ensure that adaptive learning enhances the learning experience.
**Example:**
1. **User Completes a Module and Submits Quiz with Low Score.**
2. **Adaptive Engine Recommends Revisiting the Module and Provides Additional Materials.**
3. **User Receives Real-Time Notifications and Email Recommendations.**
4. **User Engages with Recommended Materials and Improves Quiz Scores.**
Implement E2E tests using tools like **Cypress** or **Selenium** to automate these scenarios.
---
#### 10. **Ensuring Privacy and Ethical Considerations**
While implementing adaptive learning, it's essential to handle user data responsibly and ethically.
**a. **Data Privacy:**
- **Anonymization:** Where possible, anonymize user data to protect identities.
- **Secure Storage:** Ensure that all collected data, especially sensitive information, is stored securely with encryption.
- **Access Control:** Restrict data access to authorized personnel only.
**b. **Transparency:**
- **User Consent:** Inform users about the data being collected and its purposes.
- **Data Usage Policies:** Clearly outline how user data will be used to adapt learning experiences.
**c. **Bias Mitigation:**
- **Algorithmic Fairness:** Ensure that adaptive algorithms do not inadvertently favor or disadvantage specific user groups.
- **Regular Audits:** Periodically review algorithms for fairness and effectiveness.
**d. **Compliance:**
- **Regulatory Adherence:** Comply with data protection regulations like GDPR, CCPA, or others relevant to your user base.
---
#### 11. **Summary**
In **Step 6: Implement Adaptive Learning Algorithms**, we've achieved the following:
- **Defined Adaptive Learning Objectives:** Established goals for personalization, efficiency, and engagement.
- **Designed Adaptive Architecture:** Structured the system to support data collection, analysis, and content adaptation.
- **Implemented Data Collection Enhancements:** Expanded progress tracking to capture detailed user interactions and metrics.
- **Developed Rule-Based Adaptation Algorithms:** Created a rule-based engine to adjust learning paths based on user performance.
- **Integrated Adaptive Learning with Existing Services:** Seamlessly connected adaptive functionalities with course management and progress tracking.
- **Notified Users of Adaptations:** Enabled real-time and email notifications to inform users about recommendations.
- **Monitored and Improved Algorithms:** Set up logging and feedback mechanisms to refine adaptive strategies continuously.
- **Ensured Privacy and Ethical Handling of Data:** Adhered to best practices in data privacy, security, and ethical considerations.
- **Conducted Comprehensive Testing:** Validated the functionality and effectiveness of adaptive learning through unit, integration, and E2E tests.
---
### **Next Steps**
With adaptive learning algorithms in place, the next phase is **Step 7: Video Streaming Integration**, where we'll ensure seamless delivery of video content to users. This includes optimizing video delivery, handling different formats, and integrating with content delivery networks (CDNs) for scalability and performance.
Feel free to ask any questions or request further assistance regarding **Step 6: Implement Adaptive Learning Algorithms**!