I think you are looking at llms in to large a context. Your issues you have with it are the same as search and if used as a further abstraction of search it is going to carry forward weaknesses. LLM’s trained more narrowly for specific purposes are going to do better if their narrow training data is of high quality rather than throwing everything at it.