r/ProgrammingPals • u/NullPointerLeo • 23d ago

16yo backend dev – building a self-hosted RAG engine for codebases. Does this make sense?

/r/SideProject/comments/1t8nlv9/16yo_backend_dev_building_a_selfhosted_rag_engine/
1 Upvotes

100% Upvoted

2 comments sorted by

1

u/Organic_Scarcity_495 22d ago

sick project — self-hosted RAG for codebases is exactly the kind of thing that needs real infra thought. what vector db are you using? indexing approach for code (chunk by function/file/class)?

1

u/NullPointerLeo 22d ago

For the database, I opted for Chroma; it’s my first real app with AI features. I was thinking of organizing the code into blocks of classes, functions, etc. When I index a new file, each file has a path, so based on that, I have an up-to-date index of the project’s directory tree. Then the idea would be to differentiate between two types of projects later on: code-only (pure codebase) and source-only (PDFs, handouts, lectures), and perhaps a hybrid version. That way, each project is optimized for its specific type of sources. Regardless, chunking and embedding—typical of RAG systems—are performed, along with retrieval based on the similarity of the embeddings to the query. I’m not sure if I should use other approaches as well. Also, I’m pretty unsure. The architecture called for a main Spring API that ingests data and then passes it to the AI engine, which creates embeddings and calls the LLM model. Obviously, since there are two versions, I’d like to create a versatile product. Two modes: self-hosted and cloud-hosted. In the self-hosted version, files are fetched via a volume from the PC, and everything runs locally, including the LLM if desired. In the cloud version, everything runs on cloud services. The only thing that remains the same is the CLI. I’m not sure if this makes sense as an architecture; in the local Spring version, it seems unnecessary to me, yet I’d like to write code where changing a single configuration property is enough to change everything.