Code Evaluation API

API Overview

The Code Evaluation API automatically analyzes and evaluates code in GitHub repositories against specified project requirements and task criteria. This API processes GitHub repository code using LlamaIndex and leverages Google's Gemini AI model to perform comprehensive assessment of code quality, functionality implementation, and task completion.

Key features include:

• Analysis of relevant code files from GitHub repositories
• Code quality evaluation based on project requirements and task criteria
• Detailed assessment reports with specific checkpoints
• Scoring of code implementation against required evidence criteria
• Improvement suggestions for incomplete or partially completed requirements

Request Format

Request Parameters

The request body must be in JSON format with the following fields:

Processing Flow

1. API receives the request and validates required parameters
2. Creates a unique evaluation ID and stores the evaluation record in the database
3. Processes the GitHub repository using LlamaIndex to extract relevant code files
4. Analyzes repository structure and generates a summary
5. Extracts files most relevant to the specified task
6. Uses Google's Gemini AI model to evaluate code against evidence criteria
7. Generates a detailed assessment report with checkpoint evaluations
8. Updates the evaluation status and results in the database
9. Returns the evaluation results

Response Format

The API response is in JSON format with the following fields:

Success Response (HTTP 200)

{
  "success": true,
  "message": "Code evaluation completed",
  "evaluationId": "uuid-string",
  "status": "completed",
  "result": {
    "rawContent": {
      "assessment": 0.85, // Score between 0.0-1.0
      "checkpoints": [
        {
          "requirement": "Evidence item 1", 
          "status": "Completed", // "Completed", "Partially completed", or "Not completed"
          "details": "Explanation of evaluation..."
        },
        {
          "requirement": "Evidence item 2",
          "status": "Partially completed",
          "details": "Explanation of evaluation..."
        }
      ],
      "summary": "Overall assessment summary with emojis",
      "improvements": [
        "Improvement 1: Specific suggestion to meet unfulfilled evidence criteria...",
        "Improvement 2: Another suggestion..."
      ]
    }
  }
}

Error Response (HTTP 4xx/5xx)

{
  "success": false,
  "message": "Error description",
  "error": "Detailed error message",
  "evaluationId": "uuid-string" // If an evaluation ID was created
}

Possible Error Status Codes

Request Example

fetch('https://tutor-new-827682634474.us-central1.run.app/api/evaluate', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    projectDetail: "This is an e-commerce website project built with React and Node.js...",
    tasks: [
      "Implement user authentication",
      "Create product display page",
      "Implement shopping cart functionality"
    ],
    currentTask: "Implement user authentication",
    evidence: "User authentication should include login, registration, and password reset functionality...",
    githubRepoUrl: "https://github.com/username/repo-name"
  })
})
  .then(response => response.json())
  .then(data => console.log(data))
  .catch(error => console.error('Error:', error));

Important Notes

• The API operates synchronously and may take time to complete processing
• GitHub repositories must be public or have valid access credentials provided
• Evaluation results are stored in the database and can be queried using the evaluation ID
• Providing detailed project descriptions and evidence criteria will result in more accurate evaluations
• The evaluation specifically focuses on the current task only, not all tasks in the list
• The assessment score (0.0-1.0) represents how well the code meets the evidence criteria