Technology

Do not expect anything factual from llms. This is the wrong use case. You can role play with them if you guide them sufficiently and they can help with sone tasks like programming if you already know what you want but want to save time writing it, but anything factual is out of their scope.

It did not simply analyze the best type of graphics card for the situation.

Yes it certainly didn't: It's a large language model, not some sort of knowledge engine. It can't analyze anything, it only generates likely text strings. I think this is still fundamentally misunderstood widely.

May I offer you a fairly convincing explanation

The “i” in LLM stands for intelligence

All AI share a central design flaw of being what people think they should return based on weighted averages of 'what people are saying' with a little randomization to spice things up. They are not designed to return factual information because they are not actually intelligent so they don't know fact from fiction.

ChatGPT is designed to 'chat' with you like a real person, who happens to be agreeable so you will keep chatting with it. Using it for any kind of fact based searching is the opposite of what it is designed to do.

I think we shouldn't expect anything other than language from a language model.

For me it is stupid to expect these machines to work any other way. They're literally designed such that they're just guessing words that make sense in a context, the whole statement then assembled from these valid tokens sometimes checked again by... another machine...

It's always going to be and always has been a bullshit generator.

I have some vague memory of lyrics, which I am trying to find the song title theyre from. I am pretty certain of the band. Google was of no use.

I asked ChatGPT. It gave me a song title. Wasn’t correct. It apologised and gave me a different one - again, incorrect. I asked it to provide the lyrics to the song it had suggested. It gave me the correct lyrics for the song it had suggested, but inserted the lyrics I had provided, randomly into the song.

I said it was wrong - it apologised, and tried again. Rinse repeat.

I feel part of the issue is LLMs feel they have to provide an answer, and can’t say it doesn’t know the answer. Which highlights a huge limitation of these systems - they can’t know if something is right or wrong. Where these systems suggest can index and parse vast amounts of data and suggest you can ask it questions about that data, fundamentally (imo) it needs to be able to say “I dont have the data to provide that answer”

Yeah? That's... how LLMs work. It doesn't KNOW anything, it's a glorified auto-fill. It knows what words look good after what's already there, it doesn't care whether anything it's saying is correct, it doesn't KNOW if it's correct. It doesn't know what correct even is. It isn't made to lie or tell the truth, those concepts are completely unknown to it's function.

LLMs like ChatGPT are explicitly and only good at composing replies that look good. They are Convincing. That's it. It will confidently and convincingly make shit up.

And you as an analytics engineer should know that already? I am using some LLMs on almost a daily basis, Gemini, OpenAI, Mistral, etc. and I know for sure that if you ask it a question about a niche topic, the chances for the LLM to hallucinate are much higher. But also to avoid hallucinating, you can use different prompt engineering techniques and ask a better question.

Another very good question to ask an LLM is what is heavier one kilogram of iron or one kilogram of feathers. A lot of LLMs are really struggling with this question and start hallucinating and invent their own weird logical process by generating completely credibly sounding but factually wrong answers.

I still think that LLMs aren't the silver bullet for everything, but they really excel in certain tasks. And we are still in the honeymoon period of AIs, similar to self-driving cars, I think at some point most of the people will realise that even this new technology has its limitations and hopefully will learn how to use it more responsibly.

I don't want to sound like an AI fanboy but it was right. It gave you minimum requirements for most VR games.

No man Sky's minimum requirements are at 1060 and 8 gigs of system RAM.

If you tell it it's wrong when it's not, it will wake s*** up to satisfy your statement. Earlier versions of the AI argued with people and it became a rather sketchy situation.

Now if you tell it it's wrong when it's wrong, It has a pretty good chance of coming back with information as to why it was wrong and the correct answer.

What would you expect from a word predictor, a knife is mostly useless for nailing, you are using them for the wrong purpose…

It's incorrect to ask chatgpt such questions in the first place. I thought we've figured that out 18 or so months ago.

Don't use them for facts, use them for assisting you with menial tasks like data entry.

This is why my most frequent use of it is brainstorming scenarios for my D&D game: it's really good at making up random bullshit.

Ok? I feel like people don't understand how these things work. It's an LLM, not a superintelligent AI. It's not programmed to produce the truth or think about the answer. It's programmed to paste a word, figure out what the most likely next word is, paste that word, and repeat. It's also programmed to follow human orders as long as those order abide by its rules. If you tell it the sky is pink, then the sky is pink.

There's no way they used Gemini and decided it's better than GPT.

I asked Gemini: "Why can great apes eat raw meat but it's not advised for humans?". It said because they have a "stronger stomach acid". I then asked "what stomach acid is stronger than HCL and which ones do apes use?". And was met with the response: "Apes do not produce or utilize acids in the way humans do for chemical processes.".

So I did some research and apes actually have almost neutral stomach acid and mainly rely on enzymes. Absolutely not trustworthy.

Yes and no. 1060 is fine for basic VR stuff. I used my Vive and Quest 2 on one.

While I'd generally agree thst they are wrong or make up incorrect info on this case it was correct.

It gave you the min specs for vr the first time and updated specs for no man's sky the second time when you asked a more specific question.

It used your prompt of a 3070 and gave a similar perf amd card.

It doesn't know the answer, it can't run the game in vr to test. It relies on information sourced and isn't magic.

You're taking the piss right? Those seem like perfectly reasonable responses.

What video card is required to use it? None, it can be used standalone.

What video card to use it streaming from your PC, at least a 580 sounds okay for some games. You seem to be expecting it to lie, and then inferring truthful information as a lie because the information you held back (which game you want) is the reason for the heavier video card requirement.

OP those minimum requirements are taken directly from the Meta Quest 3 support page.

For such questions you need to use a LLM that can search the web and summarise the top results in good quality and shows what sources are used for which parts of the answer. Something like copilot in bing.

Did you try putting "do not hallucinate" in your prompts? Apparently it works.

Most times what I get when asking it coding questions is a half-baked response that has a logic error or five in it.

Once I query it about one of those errors it replies with, "You're right, X should be Y because of (technical reason Z). Here's the updated code that fixes it".

It will then give me some code that does actually work, but does dumb things, like recalculating complex but static values inside a loop. When I ask if there's any performance improvements it can do, suddenly it's full of helpful ways to improve the code that can make it run 10 to 100 times faster and fix those issues. Apparently if I want performant code, I have to explicitly ask for it.

For some things it will offer solutions that don't solve the issue that I raise, no matter how many different ways I phrase the issue and try and coax it towards a solution. At that point, it basically can't, and it gets bogged down to minor alterations that don't really achieve anything.

Sometimes when it hits that point I can say "start again, and use (this methodology)" and it will suddenly hit upon a solution that's workable.

So basically, right now it's good for regurgitating some statistically plausible information that can be further refined with a couple of good questions from your side.

Of course, for that to work you have to know the domain you're working in fairly well already otherwise you're shit out of luck.

This is an issue with all models, also the paid ones and its actually much worse then in the example where you at least expressed not being happy with the initial result.

My biggest road block with AI is that i ask a minor clarifying question. “Why did you do this in that way?” Expecting a genuine answer and being met with “i am so sorry here is some rubbish instead. “

My guess is this has to do with the fact that llms cannot actually reason so they also cannot provide honest clarification about their own steps, at best they can observe there own output and generate a possible explanation to it. That would actually be good enough for me but instead it collapses into a pattern where any questioning is labeled as critique with logical follow up for its assistant program is to apologize and try again.

You asked a generic machine a generic question and it gave you an extremely generic response. What did you expect? There was no context. It should have asked you more questions about what you’ll be doing.