Wednesday, November 05, 2025

What is dimensionality in data embedding

I don't want you bombardment of all the technical details pertaining to the Data chunk, embedding in the RAG system learning path or developing the AI application.

But the "Dimensionality"  of the data, a word or a sentence is very important for you to understand how your LLM model is smart enough to treat and contextualize the subsequent converstation or interaction with AI Models.

simple understanding analogue is "About me/location or any thing".

Example : evaluating the subject for the job.

Subject's name, age, sex, location and skillset all become a "feature of your data".

Consider its single array or the list in the Python terminology.

Single Array or the List is a static value and will not allow you to validate the subject job readiness profile.

So bring the more feature/attributes to the table and correlate with multiple parameters like height, weight, skillset comparison, ability to handle the project/ task, his contribution to project/team, mindset etc... with  to evaluate thoroughly.

So multiple array or dimension of the subject will gives the full picture to the user.