Policies for Using AI / LLMs

AI
1

LLMs at 0xide. By Bryan Cantrill

Article : 576 - Using LLMs at Oxide / RFD / Oxide

576 - Using LLMs at Oxide / RFD / Oxide

Using LLMs at Oxide | Hacker News

Using LLMs at Oxide | Hacker News

2

MS ‘wisdom’
https://www.thurrott.com/dev/330980/microsoft-to-replace-all-c-c-code-with-rust-by-2030

3

No mention of C# - yet.

4

No way they will replace C# - most of Azure runs on it.

C# is also 1000% safer than C - yes you can do some silly things with it, but idiomatic C# is pretty safe (much safer than Delphi) - since you don’t play with pointers - and the tooling provides a lot of help with avoiding potential oopsies.

Not getting excited about anything windows these days - the enshitified version of windows that they are slowly shoving down our throats is turning me more towards MacOS and Linux already.

5

Not so:

Microsoft Distinguished Engineer Galen Hunt posted about his ambitious goal on LinkedIn four days ago, provoking a wave of excitement and concern.

Now he’s been forced to clarify: “My team’s project is a research project. We are building tech to make migration from language to language possible,” he wrote in an update to his LinkedIn post. His intent, he said, was to find like-minded engineers, “not to set a new strategy for Windows 11+ or to imply that Rust is an endpoint.”

Research only. Likely to not go as planned IMHO. You do research to see if things ARE as you THINK. Most research shows otherwise…

1 Like
6

I might add that it’s typical of people blowing their own trumpet on LinkedIn. You don’t have to back up an aspirational post with any real-world results. If it fails it will never be mentioned again…

7

The other day, my client contacted me and asked if I could change the code to let Claude login and peruse the SPA (Single Page Application), because it was getting logged out.

I was aghast that he would even think it’s a good idea. I had put rudimentary anti-bot measures in the code itself. My main security was that not even the login page is indexed by Google. The landing page supplies the URL for the login page dynamically. The login page, if login was successful, fetches the URLs for the SPA and the REST server from the Authentication server.

While the URLs are static at the moment, I have plans to make them dynamic.

  1. I explained to my client that this was a bad idea. To drive it home, I asked Claude the honest question if this was a good idea. It said “Mostly No” with a variety of reasons.
  2. I then asked my Client if he had turned off “Used for Training”, the answer was no. I asked around, and it seems that almost no one has turned it off.
  3. Next I asked Claude to look at my security measures. It said I was using the right strategy. It found a couple of endpoints where I had forgotten to put authentication criteria. Then it gave me a list of measures (inside knowledge) to beat bots, including Claude. I am implementing them now.
  4. I asked my client to draft up a company policy and get each staff member to sign it, promising not to use any AI to access the site.

And I have posted an article on LinkedIn Stop AI Training on Your Data. It also spells out a few dangers of allowing the training to happen.

The safest way is to have your own AI.

  1. From the Symposium, I was given the impression that Olama ran fine on old laptops.
  2. I bought a NUC with 16G memory - it was too slow to be usable.
  3. I have updated it to 64G, and its passable.
  4. Someone mentioned here that they had 256G, I think that would have been ideal. Sadly my NUC cant use any more RAM.
  5. It took a lot of head scratching to get it to scan the web. That’s working well now.
  6. Next step is to get it to access my filesystem, then I cna start using it for some coding.

The major flaw in Code Desktop

They should have used a dedicated user (say Claude). Then I could have made it impossible for it to access private areas. As it stands, I have to trust Anthropic that it’s going to stay in it’s zone.

Does anyone know if this is possible?

8

For local LLM usage, it’s not the system memory that matters (well mostly) - it’s the video ram available to the graphics cards.

That and you do need a fairly beefy graphics cards, most NUC’s have integrated graphis which are pretty much useless for LLM work.

9

Yes, I hadn’t realised the full impact of that when I bought it :slight_smile:

10

Here are a few things for you to consider.

  • An option is to use a VM to run claude in. This allows you to have a GUI session and protect it from the other private data you don’t want it to access. This is probably the best and most secure solution. If you have that NUC with lots of memory that will be ideal for running the VM’s and lots of memory to run the tests and other things that Claude runs.

  • I log into remote machines using ssh. You could create that separate limited account on your local machine log into it via ssh and run claude under that. Then turn on /remote-control and you can either control it via the web interface, or run the UI as you normal user and connect to that “remote session” that is running as a different user - but with the same Claude account. This would work ok for console apps… but would be a bit awkward for GUI apps - you wouldn’t be able to get Claude to automatically start the GUI app as the user doesn’t have a console session to launch into.

  • You can use Claude Code hooks that run before a command occurs and blocks access to disallowed commands and directories. Hooks reference - Claude Code Docs

  • There is also permissions that you can manually configure. See Configure permissions - Claude Code Docs

  • Put in your claude.md file that it is not to access your private files. This works for the most part… but not guaranteed.

  • You can look at the logs of what it has done under c:\Users\username.claude\

2 Likes