Incremental Trust with Constrained 'Intelligence'

This post is focused on AI where correctness matters (financial software, Actuarial models, etc.).

It is a really exciting time to be building new technology, and for someone who builds a lot of software the changes have been, at times, jarring. Old ways of doing things are being constantly challenged and, although it’s exciting, it’s also exhausting…

We are constantly being bombarded with new models, new development processes, new tools, etc. It’s been mentally draining. Some people seem to have embraced AI wholeheartedly and these early adopters are pushing as fast as possible with software factories, etc. Others, though they are reducing in number over time, are resisting the changes aggressively.

I’m somewhere in the middle toward the embracing side, I’m excited as there are so many things I can build now, but there are parts of me that still resist. But why do I resist?

Trust & Accountability

Some people believe that software engineering is “solved”. I’m not one of those people but it is absolutely true that the cost of creating software has gone down for everyone. Previously, engineers were highly differentiated as they were skilled at the act of writing code as well as the other parts of software engineering. In a world where the value of being skilled at writing code is diminished, what is a software engineer’s value in the eyes of a user/customer?

Trust & Accountability

Trust that the software delivered represents a solid foundation to build on, that it works, that it’s sound from a security perspective, etc. However, most important is that there is trust that someone is accountable for the software. AI is many things but one thing it can never be is accountable for its output.

Trust is something that is built up over time and can be destroyed very quickly. Actuaries enjoy a large amount of trust that is derived from our membership in professional bodies like the SOA. We have to do exams, we have professional standards, disciplinary processes, etc. Actuaries should not squander this trust; there are many very talented software engineers who don’t enjoy this kind of baseline level of trust.

The culture of programming is very different from that of Actuaries. The Actuarial profession can seem downright boring compared to stories of companies built from garages and mantras like “move fast and break things”, what I will call the Silicon Valley culture…

It’s interesting to me that this culture that created so much is now a kind of obstacle for software engineers, and that they would benefit greatly from a “boring” professional body that represents a baseline of quality, a signal to the market that instills trust and accountability and would differentiate them from so-called “vibe coders”.

The Source of My Resistance

Can you trust AI to build correct Actuarial models? Probably not, but maybe. Can AI be accountable for business decisions made as a result of such models? No, and this is not something a “better” LLM, as currently designed, can solve.

I think SaaS, and Actuarial modeling software, has a future because businesses require trust and accountability, and it’s convenient to pay for this guarantee of quality (SLAs, etc.) vs the alternative.

I realized that on a smaller scale I need this trust also and it’s the source of my resistance. For me, this means trust in the tools I use and processes I rely on because, at the end of the day, if I can’t fully trust my tools and processes, how can I expect others to trust and rely on my work?

The reason for my resistance is baked into the nature of LLMs. I struggle to trust any system whose input is natural language and whose output is probabilistic. Of course, there are tasks where this is fine, but I’m talking about technical Actuarial or software-related tasks where correctness matters.

Building Something I Can Trust

Why I Was Excited About Jev

I posted on LinkedIn recently that I was very excited about the release of Jev. At the time I posted I had not used Jev, and I still haven’t. The reason I was excited was that Jev aligned nicely with a philosophy that I have in my head around how I want to approach my core tools and processes. I have been exploring some ideas in my spare time recently that require a Jev-like model. However, Jev does not fulfill all my criteria so I am building my own. I mentioned before the concept of a deterministic foundation.

I’m exploring an approach to AI adoption for “mission critical” tasks that finally feels right. I would describe it as a process of incremental trust with constrained intelligence.

Incremental Trust

As stated, the cost of creating software has gone way down. Contrary to the panic of some engineers, I think it might be the best time to be an experienced software engineer. You can build anything you like and you know how to build it in a way that you can continue to scale and build on top of. You are only limited by your ambition / ideas (and cheap tokens while they last 😄). As an aside, I’m more worried for new engineers and graduates.

I am now building small tools for many of the things I routinely do — anything that is a deterministic process is fair game. As these are tools I build myself, I can trust them. Over time, I am slowly building this “deterministic foundation”, this foundation that I fully trust.

Constrained ‘Intelligence’

Of course, there is nothing really new here other than the fact that building your own tools is faster than before. However, I’m seeking to introduce “intelligence” only where it is needed in the narrowest way possible, in the Constrained Intelligence area of the pyramid above. This is the area where I need AI to interpret loosely defined requests and make decisions about what should be used from the foundation to complete the task.

However, I still seek to trust this part as much as possible. Trust in this case does not only mean that I trust it will do what it is supposed to do. I have other qualities that I strive for when it comes to key tools/processes:

• I don’t want to rely on the current cost of tokens
• I don’t want to have to pay a subscription
• I can’t risk getting cut-off from a tool I rely on
• I highly prefer tools that run locally

The requirement around local tools is twofold. I like local tools with very low latency that can run offline; this is personal preference. However, working in an industry like insurance means tight restrictions on data at times; I can’t assume I can share data with any cloud provider or service. At the moment, this does not matter as this is a personal experiment but I’m building toward the future and I hope to adopt these tools/processes at work also (I also enjoy the technical challenge of building local-only tools 😄).

lowpass Is Open Source

One reason I have not been using Jev is that I have been building something similar for myself that meets all my requirements. I recently split out one component from my “constrained intelligence” area and I open sourced it as lowpass. I plan to open source more as I progress.

lowpass is a 100% local, low-latency, and deterministic tool-calling framework. Instead of relying on generative Large Language Models (LLMs) this framework utilizes a Joint Intent Classification and Slot Filling approach.

It is similar in the sense that it is designed to take natural language and return tool calls. Its scope and requirements are different; you can read more on the GitHub page.

I’m using this foundation, with some other tools not yet open sourced, to implement the top of the pyramid. Each lowpass is small and trained for a very narrow scope, but because they are cheap to execute, I can combine them in specific ways that suit my workflow to build the kind of “intelligence” I need. lowpass can be re-trained in seconds, so I can have applications continuously adapt to my speech patterns from log data so that they understand me. In a sense, I don’t need a frontier model that is trained on a lot of data so that it can understand anyone because no one else matters. As a result, I don’t pay frontier model latency or tokens.

Conclusion

I’m still experimenting with state-of-the-art models and new experimental things all the time. My choice of tool very much depends on the task I am completing. However, I’m enjoying experimenting with the process of incrementally building a foundation of trust; I feel like I’m building something that will serve me regardless of whether an AI bubble bursts or not. In a way, this reassurance has freed me to lean in even more and experiment more with all the new innovations in AI.

I feel there is a parallel to AI strategy here for any business in a highly regulated environment. It’s not a complete AI strategy but part of one. There is great value in looking at your current business processes and applying a similar philosophy. Use the current tools (and subsidized tokens) to build and incrementally improve a foundation you trust, then deploy just enough intelligence to leverage that foundation and unlock business value.