Skip to main content

Project Setup, Dependencies, and Core Tasks

This guide provides a comprehensive overview of all dependencies, installation commands, and the first two core project tasks as defined in the Task-master system. Follow these steps to ensure a production-ready, fully operational environment.

1. Dependencies​

System Requirements​

  • Python: 3.11 or higher
  • Docker & Docker Compose: 20.10+
  • Node.js & npm: Node.js 18+, npm 8+
  • Git
  • GNU Make
  • UV (Python package installer)

Python Dependencies​

All Python dependencies are listed in requirements.txt. Key packages include:

  • FastAPI, Uvicorn, Streamlit
  • pymilvus, neo4j, redis
  • langchain, transformers, spacy, nltk
  • pandas, numpy, scikit-learn, matplotlib, plotly
  • pytest, black, mypy, flake8, jupyter

Install with:

uv pip install -r requirements.txt
uv pip install -r requirements-dev.txt

Node.js/Frontend Dependencies​

  • Docusaurus (documentation site)
  • All dependencies listed in docs-site/package.json

Install with:

cd docs-site && npm install && cd ..

Other Tools​

  • Pre-commit hooks: pre-commit install
  • Docker services: docker-compose (see Makefile)

2. Installation & Setup​

Initial Setup​

git clone <repository-url> smart_rag
cd smart_rag
cp .env.example .env
cp .claude/settings.local.json.template .claude/settings.local.json

Edit .env and .claude/settings.local.json as needed.

Install All Dependencies​

uv pip install -r requirements.txt
uv pip install -r requirements-dev.txt
cd frontend && npm install && cd ..
cd docs-site && npm install && cd ..
pre-commit install

Start All Services​

make services-up
make health-check

3. Task 1: Complete Project Infrastructure Setup​

Objective: Finalize project infrastructure with all required services and development environment.

Steps:

  1. Verify all Docker services are properly configured (PostgreSQL, Milvus, Neo4j, Redis, MinIO)
  2. Complete GitHub Actions CI/CD pipeline setup
  3. Configure Kubernetes manifests for deployment
  4. Set up monitoring stack (Prometheus/Grafana)
  5. Implement health check scripts
  6. Configure Claude Code environment

Test Strategy:

  • Run: 
    make services-up
    make health-check
    make test
  • Verify all services are healthy and CI pipeline passes.

Subtasks:

  • Docker service configuration: docker-compose.yml, docker/ scripts
  • CI/CD: .github/workflows/ci.yml
  • Kubernetes: manifests in infrastructure/
  • Monitoring: Prometheus/Grafana via Docker
  • Health checks: scripts/health_check.sh

4. Task 2: Implement Core RAG System Components​

Objective: Build the core Hybrid RAG system with vector, graph, and orchestration components.

Steps:

  1. Complete VectorStore implementation with Milvus
  2. Implement GraphStore with Neo4j integration
  3. Build HybridOrchestrator for query routing
  4. Implement EntityExtractor using spaCy/HuggingFace
  5. Create ContextEnhancement module
  6. Build UserProfileStore for personalization

Test Strategy:

  • Run: 
    pytest src/graphrag/rag_system/test_integration.py
  • Verify all components work together as expected.

Subtasks:

  • VectorStore: CRUD operations, connection pooling, error handling
  • GraphStore: entity creation, relationship mapping, graph traversal
  • HybridOrchestrator: query routing, result fusion, ranking
  • Entity Extraction: spaCy/HuggingFace integration
  • Context Enhancement: user profile-based query enhancement

5. Verification & Troubleshooting​

  • See SETUP_VERIFICATION.md for detailed verification steps and troubleshooting tips.
  • Use make help for a list of all available Makefile commands.
  • Review CI/CD logs for pipeline issues.

6. Subtask Documentation​

7. System Architecture​

flowchart TD
  subgraph Infrastructure
    DB1[PostgreSQL]
    VEC[Milvus]
    GDB[Neo4j]
    RED[Redis]
    MIN[MinIO]
    MON[Prometheus/Grafana]
  end
  subgraph CI_CD[CI/CD Pipeline]
    GH[GitHub Actions]
    K8S[Kubernetes]
  end
  subgraph RAG[Hybrid RAG System]
    VS[VectorStore]
    GS[GraphStore]
    HO[HybridOrchestrator]
    EE[EntityExtractor]
    CE[ContextEnhancement]
    UP[UserProfileStore]
  end
  GH -->|Build/Test/Deploy| K8S
  K8S -->|Deploys| VS
  K8S --> GS
  K8S --> HO
  K8S --> EE
  K8S --> CE
  K8S --> UP
  VS --> VEC
  GS --> GDB
  HO --> VS
  HO --> GS
  EE --> CE
  CE --> UP
  VS --> DB1
  GS --> DB1
  VS --> RED
  GS --> RED
  VS --> MIN
  GS --> MIN
  MON --> GH
  MON --> K8S
  MON --> VS
  MON --> GS

For further details, consult the Setup and Verification Guide and the Makefile.