9

I tried using the LLMFunction in Mathematica 14.0 using e.g. the example given in its documentation.

Before this, I logged into my Wolfram account as suggested by @Nasser here and provided the key for my free/unpaid OpenAI account.

I consistently receive the error message below. This suggests I cannot use the Mathematica LLMFunction without having a paid OpenAI account with available tokens. However, both the Mathematica documentation and the quoted answer by Nasser are unclear on this.

Can anybody please let me know if they have been able to use the Mathematica LLM functionality without a paid OpenAI account (without providing payment details), and how? Is this possible at all?

enter image description here

The link given in the Mathematica error message, given below, suggests indeed that one needs to have a paid account. If this is the case, I am surprised this fact is not clearly highlighted in the Mathematica documentation.

enter image description here

divenex
  • 616
  • 5
  • 12
  • 5
    Well ... The second bullet point in the documentation for LLMFunction (Details and Options) states: "LLMFunction requires external service authentication, billing and internet connectivity." ... – Domen Jan 15 '24 at 11:49
  • 3
    That's quite a hidden and ambiguous manner to state that to use a key Mathematica functionality one requires to subscribe to a service, in addition to the Mathematica license! This means any person without a credit card, or unwilling to subscribe to OpenAI, cannot use some of the key innovations of the latest versions of Mathematica. I hope others can confirm. – divenex Jan 15 '24 at 11:59
  • 3
    Personally, I would only bother trying it out I could configure it to use ollama or some other offline solution. I won't accept OpenAI's terms of use and privacy policy, or pay them, so this whole block of Mathematica functionality is off limits to me for the time being. Also there's a bug in the Preferences > AI Settings. If you don't supply an API key and just click the [x], it messes up the UI. However, after some attempts, it eventually went away and revealed that you can change the service to Anthropic or PaLM, whatever they are, but no option to use an offline service. – flinty Jan 15 '24 at 12:22
  • There may be some way to run an offline llm, and use your hosts file / dns to force the traffic to go to your own service instead which just pretends to be openai and ignores the api key, but you'd need to figure out where to get the models / prompts etc. – flinty Jan 15 '24 at 12:28
  • Strictly speaking: you don't need an OpenAI account...as you can use the LLMFunction, TextSummarize, and LLMSynthesize functions using Anthropic and (Google) PaLM https://reference.wolfram.com/language/guide/LLMFunctions.html – Joshua Schrier Jan 15 '24 at 22:57
  • 2
    @flinty has a fun idea about faking OpenAI. I'm aware of a project to do this (you'd have to also modify hosts) https://localai.io Perhaps also worth noting is a discussion at community have demo'd playing with ollama, etc. https://community.wolfram.com/groups/-/m/t/3097299 (albeit without using the "built in" functions) – Joshua Schrier Jan 15 '24 at 23:06
  • You can get an API key for other models basically for free. Then use LLMEvaluator -> LLMConfiguration["Model" -> "gpt-3.5-turbo"] as an option in LLMFunction. – Sam Gralla Jan 29 '24 at 03:59

1 Answers1

9

According to Wolfram documentation, the following four functions say the following in the help page for each

requires external service authentication, billing and internet connectivity.

These are the functions

ImageSynthesize used to generate a synthetic image based on a textual description or a reference image

LLMFunction used to generate text using a large language model (LLM)

TextSummarize used to condense textual information.

LLMResourceFunction can retrieve resources stored locally, in the cloud or in the Wolfram Prompt Repository.

Nasser
  • 143,286
  • 11
  • 154
  • 359
  • I interpret this as a YES: To use any LLM functionality one needs an OpenAI subscription and must pay extra for every function call. This makes it useless for most people. It is disappointing that this is not stated more explicitly in the documentation. – divenex Jan 17 '24 at 10:44