Has anyone tried aikido.dev for security scanning their Delphi apps? It’s a high end commercial tool that claims to offer a lot of features (static and dynamic, pentest) and supports Delphi.
We had a product demo of aikido (which I wasn’t included in) and have been asked if it would be useful for our Delphi based products (we have many other products in Java, C#, Ruby that are likely to have good support so we may roll it out across all products).
We trialled DerScanner but that was very light on support for Delphi (they double up the number of metrics because they count Delphi and Pascal separately, lots of false positives, limited depth). We are using SonarQube but for Delphi it only includes quality metrics, not security (and the open source Delphi metrics can be used independently in DelphiLint as demoed by Alister at the Symposium).
Sadly I don’t think there are really any products out there that have comprehensive security scanning for Delphi. It’s just too much work for too small a market for the big players.
There’s probably gap in market for someone brave enough to tackle it - I’m not that brave.
LLMs seem quite good at SAST (static testing), at least that is my experience with a Python based REST API. It is quite a bit slower than regular SAST tools but I think that it would find issues both more broadly and deeply. Aside from just a best practice review you can ask the model to review based on the OWASP top N, or provide a list of the ASVS rules that you want to focus on. A lot of the rules are focused on web apps so some of them don’t apply to desktop apps which also have their own unique challenges. I liked Alister’s talk on desktop app security at last year’s Symposium - that included some fun issues to consider .
I’m leaning into LLM’s a lot for security when it comes to web applications - my current stack is tanstack start using bun which is all typescript/react based. Claude does an amazing job of helping to lock things down. I make it a point to have it do a security audit when ever
a) a new model is released
b) I have been making changes
I did just this yesterday on my package server - whilst claude didn’t find any major issues, it did find that spammers could use the contact form to spam package owners. IT was rate limited already, but it it wasn’t sanitising the input so spammers could send links etc.
In the past it found all sorts of issues - mostly minor but still worth addressing.
A static code analyser that fully understands delphi would be nice - but lacking that, LLM’s are what we have - I just wish I could afford the hardware needed to run everything locally so I’m not training claude on my code.
With the LLM based security review I had it categorise each potential issue as critical, high, medium, low, info which helped to focus on the more important issues.
One thing that LLMs seem to miss regularly is choosing out of date packages even sometimes when you ask for the latest stable releases - I have to ask the LLM to check the web for the latest version otherwise it picks the latest that it was aware of in its training data.
Claude shouldn’t be training on your code unless you’re just using the web app - at least check the config in your account. By default your code/docs are not used for training in Claude Code or other agentic tools that use the LLM APIs directly. That being said, every other month there seems to be a release of a local model that climbs the AI coding leaderboard. Ian posted a link to llmfit in the “new workflow” thread, which is a tool that you can run locally to tell you which models your hardware is likely able to use. That would be rather useful.
How do we really know? The models are getting better and better, they know more about delphi code every release - I don’t trust these cloud based ai companies - they are suddenly finding new data to train on that they couldn’t find before?
I use claude code extensively, on my open source projects. For my product code, I only show it what I need to, never the full project. That does limit it’s usefulness, but I’m willing to live with that for now.
I am a bit more trusting but you do have a point. If any provider were found to have trained on confidential data there would likely be severe repercussions. I don’t think they would take that chance.
On the point of getting better at Delphi code I have noticed that too - at least when I include instructions to use modern Delphi syntax, inline variables, type inference, generics collections, etc the models seem to be better at sticking to that. Perhaps it is a matter of the LLM providers tweaking the weights of source code in their models, or ingesting more open source code.
@ianbarker I wonder if it would be possible to open source the interface sections of all RTL/VCL/FMX source, as-is or auto sanitised to some degree such as extract type defs, protected/public/published sections of classes and records and all constants, to expose that to LLMs in future training to improve Delphi code gen.
.