Yeah, OP clearly hasn’t used LLMs very much, they can absolutely produce the text necessary for those things. Not reliably right now, mind you - people have been fined for citing made-up case law because ChatGPT went rogue - but it can do a great first pass.

Passing the test is not the same as being admitted to practice law. Can you give me the bar ID number of a non-human AI who has sat for and passed the bar in a US state? Last time I took it it required having a face, a social security number and a JD.

I think passing a standardized test and legal research and writing are almost certainly things that AI can do better than any human being. But that’s a very different statement than saying that people should stop going to law school because AI is being developed. I know if I was facing jail time I would prefer to have a lawyer with biological neurons.

I think less people should go to law school because being a lawyer sucks ass, but that’s a different discussion.