Binny Gill, Founder & CEO Of Kognitos – Interview Series

Binny Gill has a diverse and extensive work experience spanning multiple roles and companies. Binny is currently the Founder and CEO of Kognitos, a company focused on making programming accessible and enabling businesses to optimize their operations and customer experiences.

Binny's a prolific inventor in computer science, with close to 100 patents, and believes that more people need to be able to instruct computers in natural Language.

Could you share the genesis story behind Kognitos?

During the pandemic, my son decided to make the game of tic-tac-toe in Python. He built it in a couple of days, and I was a proud dad. However, I woke up the next day realizing that I had made the same game in about the same amount of time 30 years ago. I was the same age then. It dawned on me that programming has not become any easier over the decades. All that we have done is made more humans understand programming.

I went back to challenge my son to write another program. This time to find out if a number is prime or not. I found myself trying to teach programming by saying that he needed to "think like a machine". That didn't go anywhere. Then I realized what I was missing. I taught him to first write "the pseudo code" (just an explanation of what the program will do but in his own words). That was easy, it took 5 minutes. We started converting that into working code. It was hard for a first-time programmer and after a few hours my son said he didn't want to code any more.

I was taken aback. Why was programming so hard even after 7 decades of innovation and thousands of programming languages being invented? I offered my son that I will find a language that works for him. He immediately said, "why can't this work?" — he was pointing to the pseudo code he had written in 5 minutes for the prime number problem. I laughed and said, "No, those are just your notes. The machine can't understand that".

"Why can't it be like Alexa?", he said incredulously. And that was a light bulb moment. After a long silence I told my son to not learn Python. Kognitos was born.

Can you dive into the inner workings of the platform? How is Kognitos catering to customers?

Kognitos is the world's first automation platform built entirely in English. We have built a first of a kind interpreter for natural language that understands and executes natural language code. The impact of this is huge as now all business users, whether highly technical developers, or financial analysts, or high-school graduates processing invoices all can understand and use the same automation tool.

From the business perspective, the impact occurs in several areas. The time required to build automation is reduced as there is no necessary translation from English steps to python or other coding languages. The business user is now able to use their specific functional knowledge to handle exceptions and teach Kognitos how to handle future examples. This lessens the burden on IT. And lastly, compliance and IT are happy as all of the data on what both humans and AI did is stored in English, so it's easily accessible as needed.

What are some of the machine learning algorithms that are used, and what part of the process is Generative AI?

Kognitos combines two fundamental technologies to deliver an automation platform that works in the manner of people. Just like humans have two sides of their brain, one that is highly logical, and one that uses pattern recognition and intuition to be creative, Kognitos has two sides. First, Kognitos is built on our patented interpreter, the world's first to "Run English as code". The interpreter (the logical side) provides the consistency, determinism and auditability needed for operating business processes.

We combine this with LLMs (the creative side), to enhance its capabilities and make the platform even more approachable for users. One example of this is with our conversational exception handling. When an error occurs (for example a document is missing in a workflow), Kognitos feeds the error to an LLM and instructs it to present the error in a way that the business user can understand it and respond. The user can then respond in English (like a conversation) telling Kognitos how to solve the problem. We use the best model for each situation including GPT 3.5, GPT 4, Palm 2, and others. As the business user handles exceptions, the system is learning from these examples and using a few prompting techniques can quickly understand what the business user does without the need for extensive training, as used to be case with traditional AI models.

How does Kognitos differentiate itself from competition? How is it used at the enterprise level?

Kognitos differentiates itself by removing the need for highly trained developers or data scientists, and in doing so eliminating much of the maintenance cost in automation. RPA developers are not only expensive but also in short supply. This results in competitive products (which are primarily built on early 2000s technology), long backlogs of unfinished projects in IT, software on the shelf, and high maintenance costs for what is already implemented.

Because Kognitos democratizes automation by making it accessible to everyone in the language of business, English, now business users are able to be involved in the automation process. Organizations may still want more technical users to build the automations as a part of their governance process, but the handling of exceptions shifts to the business users who have the subject matter knowledge to handle them. This greatly reduces the costs of all automations, creating strong ROI cases for automations that previously were not viable with RPA. As a result, businesses primarily use Kognitos for processes that are high-volume, repetitive, manual, and contain lots of exceptions or variations. Commonly these processes are found in Finance, Accounting, HR and supply chain.

How did your background in cloud software influence your vision for Kognitos? What are the areas of overlap between cloud and generative AI?

My vision is to bring computer literacy to the masses – not by forcing more humans to speak the language of the machines, but by upskilling machines to speak the language of humans. All my life I have spent learning myriad computer languages and have always felt that the experience of programming has been suboptimal. Why can't the machine ask me a simple question instead of crashing in the middle of a long automated process? I believe that the paradigm of programming (be it cloud or be it process automation or AI) is fundamentally shifting today to natural language.

Ever since we moved from punch cards and assembly programming to C, Fortran and Cobol, there has not been any fundamental improvement in programming languages until now. We are now moving from the realm of precise languages for programming computers to imprecise languages for programming then using natural languages. The reason why this is becoming possible now is because machines are finally able to talk back to the human to clarify the intent of the program. That is huge and will impact all of computer science (not just cloud but every piece of software around us). I believe all business apps will now be written in English.

How does Kognitos prioritize human oversight while leveraging rapid advancements in AI?

In the industrial age, we built machines much more powerful than us and relieved people of manual labor. The key element to making it safe was that we humans had the "steering wheel" in our hand to control the machine. With the rapid advancements of AI, we are now entering the era when we will be building machines much more powerful than us which will relieve us of mental labor. However, where is our new "steering wheel"?

At Kognitos, we believe that steering wheel is the democratization of automation review. While we harness the creativity of LLMs to write automations, making it possible for all humans to review those automations is the key to remaining safe and in control. By providing a platform where what the machine plans to run deterministically is expressed in natural language, Kognitos is giving most of humanity that much needed "steering wheel".

Just like the human brain, the Kognitos interpreter is dualistic in nature (Logic + LLM). Logic is the antidote for hallucinations, and by building the LLM layer on top of the logical interpreter, Kognitos is able to enforce validations in a deterministic manner after any LLM-based step that requires review. Further, being a stateful system, the Kognitos platform records every action of both the human and AI in English and thus is a 100% auditable and whitebox AI system.

At the moment, most business activities are done via computers and mobile devices. What needs to change before businesses truly embrace new technologies like augmented reality and virtual reality?

As we enter the era where machines pass the Turing Test, all the traditional interfaces that were invented because machines could not understand humans directly will get dismantled. Already I prefer not to open apps on my smartphone if Alexa or Siri can do the job for me. Human-Computer Interface design will give way to Human-Human Interfaces for machines. So, I foresee all drag-and-drop and menu-based interfaces giving way to natural language-based interfaces.

To answer the question as to whether augmented and virtual reality will be embraced by businesses – we first need to see that happen in the consumer world. If it isn't happening in our kitchens at home, then it is unlikely to happen on any large scale in businesses. What I foresee is a revolution in robotics following the revolution in Generative AI. Those robots will be the interface to machines both at home and in businesses. Humans like to keep things real.

What do you expect to be the next big breakthrough in AI?

The invention of artificial general intelligence (AGI) that could learn to accomplish any intellectual task that human beings can perform might happen, but as a society we should discourage that. I favor the invention of a collection of ANI (Artificial Narrow Intelligence) models that will help humanity in narrow tasks. However, by combining these ANI models via a logical and auditable system we can achieve monumental tasks while still being in control of the overall process.

What is your vision for future advancements in business process automation?

The role of humans in businesses is going to dramatically change. First business process information that in people's heads will get translated into machine code using natural language platforms like Kognitos. Once the processes are in the machine, by running those processes, the machine will start to build a business journal of everything that happens in the business. That creates a treasure-trove of data that really captures the essence of any business.

Eventually, superhuman narrow intelligence models will run each aspect of a business (from marketing to sales to engineering). That "talent" will never leave the business anymore. Humans will have a review only – almost legislative role. The humans will approve new policies and decide on ethical questions and take responsibility for business actions. However, most of the operations of the business will be done by machines.

Is there anything else that you would like to share about Kognitos?

At Kognitos we deeply care about the future safety of humanity in the presence of super-human intelligence. The collective power of humans today is expressed through the machines we have built. Those machines, be it factories or cars or war machines, are controlled by computers. Today Generative AI is writing programs to control these machines. And those programs are expressed in traditional computer languages, and it is hard to convince ourselves that there won't be any biases or hallucinations creeping into those generated programs. The only way to keep ourselves safe is to review all those programs. However, reviewing traditional programming languages requires developers and we don't have enough of them in the world.

We are currently living in the dark ages of computer literacy, with 1 in 200 people able to review any code. By changing the language of automation to English, Kognitos will allow 100x automations to be reviewed by humans, amplifying the review bandwidth of humans by orders of magnitude and keep humans safer in the presence of super-human AI.

Thank you for the great interview, readers who wish to learn more should visit Kognitos.

Forth: The Hacker's Language

Let's start right off with a controversial claim: Forth is the hacker's programming language. Coding in Forth is a little bit like writing assembly language, interactively, for a strange CPU architecture that doesn't exist. Forth is a virtual machine, an interpreted command-line, and a compiler all in one. And all of this is simple enough that it's easily capable of running in a few kilobytes of memory. When your Forth code is right, it reads just like a natural-language sentence but getting there involves a bit of puzzle solving.

From Thinking FORTH (PDF)

Forth is what you'd get if Python slept with Assembly Language: interactive, expressive, and without syntactical baggage, but still very close to the metal. Is it a high-level language or a low-level language? Yes! Or rather, it's the shortest path from one to the other. You can, and must, peek and poke directly into memory in Forth, but you can also build up a body of higher-level code fast enough that you won't mind. In my opinion, this combination of live coding and proximity to the hardware makes Forth great for exploring new microcontrollers or working them into your projects. It's a fun language to write a hardware abstraction layer in.

But Forth is also like a high-wire act; if C gives you enough rope to hang yourself, Forth is a flamethrower crawling with cobras. There is no type checking, no scope, and no separation of data and code. You can do horrible things like redefine 2 as a function that will return seven, and forever after your math won't work. (But why would you?) You can easily jump off into bad sections of memory and crash the system. You will develop a good mental model of what data is on the stack at any given time, or you will suffer. If you want a compiler to worry about code safety for you, go see Rust, Ada, or Java. You will not find it here. Forth is about simplicity and flexibility.

Being simple and flexible also means being extensible. Almost nothing is included with most Forth systems by default. If you like object-oriented style programming, for instance, Gforth comes with no fewer than three different object frameworks, and you get to choose whichever suits your problem or your style best. You can modify the Forth compiler or interpreter itself, so if you want type checking, you can add it. Some Forth implementations are written with just twenty or thirty functions in native assembly or C, and the rest is bootstrapped in Forth. Faster Forths are implemented entirely in assembly language, with some compile-time optimizations that make it run about as fast as anything else, even though it's compiled as you type, on the target microcontroller itself.

Forth is probably not the language you want to learn if you are designing an enterprise banking backend. On the other hand, a string of blinky LEDs running a physics simulation isn't an "enterprise" anything. Forth is a hacker's language, in both the laudatory and the pejorative senses. I'm not sure that it's going to help you get "real work" done at all, and you engineer types might want to walk away now. But if you want to tweak at the language itself, or use it to push your hardware around, or just play, Forth is fantastic. The hacker in me thinks that's a lot of fun, and it's a great match for smaller microcontroller projects.

Forth Crash Course: Theory Section

Forth is the simplest language after assembly language. Forth is procedural in the extreme — a Forth program is really just a chain of subroutines, called "words" in the Forth jargon. There's no syntax, and all words are separated by a space and are parsed left to right. With a few exceptions for compiling, all words run right now so the Forth interpreter doesn't have to look ahead to the next word. Run this code! Is valid Forth if you've already defined the words run, this, and code! And it calls each of the three words in the order that you'd expect.

Simplicity and Reverse Polish Notation

The corollary of this simple setup is that Forth uses the so-called Reverse Polish Notation (RPN). If the Forth interpreter is going to execute the + word right now, it has to have the two numbers that are going to get added already on the stack. Otherwise, it would have to wait for the next number to come along before it could finish the addition. Read that again, and let it sink in. It's different from what you're used to, and it's important.

Any other ordering or syntax is unnecessarily complicated. The way that you think of as "natural" to write down math is crazy, in the sense that the look-ahead nature of the + operator requires either parentheses or an "order of operations" to be unambiguous. Look at 2 + 3 * 4. There is nothing natural about getting 14 here at all — it's the result of a convoluted syntax with implicit rules. You have to read the whole "sentence" first, find the *, remember that it has priority over +, evaluate it first, and then go back to the addition to finish up, even though + is the second word in the sentence. A computer doesn't want to know about "order of operations", it just wants to add two numbers, preferably ones that are already sitting in ALU registers. Don't believe me? Read the machine code.

Forth, and RPN, write this as 2 3 4 * + or 3 4 * 2 +. Either way, the operator works on whatever numbers are already available. If you don't think of this as being "reverse" or "polish" or even a "notation", you'll be on the right track. You're simply writing things down in the order in which they should be executed. (How crazy is that!?!)

I like to think of RPN as the computing equivalent of mis en place; before you start cooking, you get all your ingredients lined up. This is the way Forth code works: get.Broccoli chop.Broccoli get.Beef slice.Beef get.Oyster-sauce stir.Fry Some elements are naturally interchangeable — you could get and slice the beef before the broccoli — but the overall order is important to the procedure, and you really don't want to be going back to slice the broccoli while the beef is in the wok. But this is exactly what you're doing when you insist on writing 3 + 4 instead of 3 4 +.

Compiling and Running

New words are defined and compiled with a : to enter compilation mode and a ; to exit. Compilation, such as it is, takes place immediately. Under the hood, the Forth interpreter is looking up each word that you use in the definition and simply stringing them together. One exception to this rule is the function name itself, as you'll see now.

Starting FORTH

Compile your first Forth word: : seven 3 4 + ;. It's not very useful, but it creates a word called seven that will put a 3 and a 4 on the stack and then run the addition word. The "result" is that whatever you had on the stack before, you'll have a 7 on top of it now. An optimizing Forth compiler will just push a 7 onto the stack.

All programming is about breaking complicated tasks down into reasonable-sized chunks. What constitutes "reasonable" depends a bit on the language, a bit on the programmer's own style, and a bit on the cultural zeitgeist. Executing a word in Forth requires a lot less overhead than a function call in C, and following long code logics can get convoluted even for experienced Forthers, so the definition of Forth words tend to be extremely short, typically one-liners including comments. You'll be compiling quite often.

The Stack

The heart and soul of Forth is the data stack, henceforth "the stack". Forth is a stack-based language, and until you've coded in Forth for a while, you can't appreciate what this really means and how thoughts about the stack come to dominate your coding life. Forth words don't take arguments or return values, instead they operate on whatever data is on the stack when they're called.

Starting FORTH

The stack is the best and worst part of Forth. When your stack contents line up right with the words that operate on them, the result is code of a beauty and efficiency that can't be beat. When they misalign, you find yourself wondering if the object-oriented folks aren't right, and that coupling data with methods might be a good idea after all.

The gimmick in Forth programming is figuring out what's needed on the stack by one word, and making sure that the word used just beforehand leaves that on the stack. In this sense, the Forth programmer defines words, but needs to think in phrases, where the contents of the stack start out empty and end that way again. This is like the way that C uses the stack within a function scope, keeping the local variables only until it's done with the function, and then overwriting them.

As a concrete example of this chaining, imagine a word, gpio-set that sets a GPIO pin high. It will probably need both a port and a pin number to get the job done. A particularly Forthy way to implement this is to define a word for each pin on the part that you're going to use: : PA3 PORTA 3 ; Then you can light the LED on pin A3 with PA3 gpio-set. In C, you'd first define a structure that includes a port and a pin number, then define gpio-set to take a structure of that type. In Forth, this is implicit: you make sure that the pin words push a port and pin number onto the stack, and then that pin-handling words expect them. It's not safe, but it's simple.

Stack Juggling Swap-swap, the Forth Mascot, from Starting FORTH

The absolutely worst part of Forth is stack manipulations. People usually start learning Forth by learning about the stack manipulations, and indeed they are important, but they're trivial. Words like swap, drop, and dup let you move items around on the stack, but too many "stack juggling" manipulations in a given word is probably a sign of bad Forth code, rather than good. You'll use these words for sure, but that's not where the bodies are buried.

Rather, the stack is where the absolute minimal amount of data wants to sit between processing steps. The number of items needs to be small enough to avoid running out of finite stack space, of course. But a second reason is that it's simply hard to keep too many stack items straight in your mind. As you get used to Forth, the amount of stack that you can internalize might go up from three items to five or seven, but you're going to get confused if you let the stack grow unpruned for long.

So Forth is a disaster. It's a language where you have to manage the stack yourself, and the only hope of not getting caught in a tangled web of endless stack juggling is to keep things as simple as possible. Forth proponents would claim that this is also its greatest virtue — there's always pressure to keep things simple and straightforward because doing anything else will embed un-figure-outable bugs so deep into your code that you'll go mad. I don't know if it's Stockholm syndrome, whether I'm a minimalist at heart, or whether I like the challenge, but this is actually one of the reasons that Forth is so neat.

Other programming languages allow you to juggle tens of variables because the compiler keeps track of different scopes for you, keeping track of how deep each (local) variable is on the stack. Forth doesn't. And that means that you need to think about the order in which things run. Forth forces you to internalize a bit of optimization that an ideal compiler would do for you. This can end up in very tight code, or in headaches. It depends on the programmer and the problem.

In my experience, simple syntax, computer-friendly ordering, and the resulting emphasis on transparency and simplicity make it actually surprisingly easy to get stuff right in Forth.

The Sweet Spot

But enough philosophical crap about Forth. If you want that, you can read it elsewhere in copious quantities. (See the glossary below.) There are three reasons that Forth more interesting to the hardware hacker right now than ever before. The first reason is that Forth was developed for the computers of the late 1970s and early 1980s, and this level of power and sophistication is just about what you find in every $3 microcontroller on the market right now. The other two reasons are intertwined, but revolve around one particular Forth implementation.

Mecrisp-Stellaris

There are a million Forths, and each one is a special snowflake. We've covered Forth for the AVRs, Forth on ARMs, and most recently Forth on an ESP8266. The joke goes that if you've seen one Forth implementation, you've seen one Forth implementation. But I think that's too cynical — Forth is as much a way of thinking about the computer programming problem as it is a particular implementation. Once you learn one, you'll be on good footing to learn any other.

Anyway, a few years ago, a physics graduate student [Matthias Koch] wrote a Forth for the MSP430 because he needed a microcontroller to collect analog data for an experiment. That's "Mecrisp". Later on, he needed more speed and re-wrote it for the ARM Cortex M family of chips, and we got "Mecrisp-Stellaris".

And that's where things got awesome. [Jean-Claude Wippler], the "J" in JeeLabs, decided that he was going to implement his new system of distributed electrical and environmental sensors in Forth, at least under the hood. To do this, he needed a hardware abstraction layer. The combination of Mecrisp-Stellaris with the JeeLabs libraries is a tremendous Forth ecosystem for a whole bunch of ARM microcontrollers, and it has all been developed within the last two years by a small core of thoughtful hackers. Combining the two provides a very pleasant Forth microcontroller hacking experience, like a weird interactive Arduino.

Your Homework

So if you want to follow along down a very strange rabbit hole, learn a bit about the real hacker's programming language, or just fool around, stay tuned. In a couple of weeks, I'll publish a hands-on guide to getting started with Mecrisp-Stellaris on the STM32 family ARM chips.

These are the droids you're looking for.

A good minimum Mecrisp-Stellaris development environment is going to consist of a cheap STM32F103 development board, a ST-Link v2 (or better) programmer for it, and a USB-TTL serial adapter. Chances are good that you've got at least the latter kicking around already. If so, you could be set up with the rest for around $10, €10, or £10 depending on where you live. "STM32F103" and "ST-link" should get you set up on eBay.

So order some parts right now. While you're waiting for delivery, work through this very good online tutorial. Run through that, and you'll be set for next time.

If you want to go further, you can download a Forth for your desktop computer and start working through some other introductions.  Brodie's "Starting FORTH" is the canonical Forth introduction, but it's a bit dated (and feel free to skip Chapter 3 on disk access). If you prefer brevity over humor, check out J.V. Noble's Beginner's Guide or the Gforth tutorial.

All images from Starting FORTH used by permission from FORTH, Inc. The originals are black and white, rather than Hackaday dark charcoal and yellow, naturally.

What Is Llama 2? Meta's Large Language Model Explained

Llama 2 is a family of pre-trained and fine-tuned large language models (LLMs), ranging in scale from 7B to 70B parameters, from the AI group at Meta, the parent company of Facebook. According to Meta AI, Llama 2 Chat LLMs are optimized for dialogue use cases and outperform open-source chat models on most benchmarks they tested. Based on Meta's human evaluations for helpfulness and safety, the company says Llama 2 may be "a suitable substitute for closed source models."

Llama 2, like the original Llama model, is based on the Google transformer architecture, with improvements. Llama's improvements include RMSNorm pre-normalization, inspired by GPT-3; a SwiGLU activation function, inspired by Google's PaLM; multi-query attention instead of multi-head attention; and rotary positional embeddings (RoPE), inspired by GPT Neo. Llama training used the AdamW optimizer. Llama 2's primary differences from Llama are increased context length (4096 vs. 2048 tokens) and grouped-query attention (GQA) instead of multi-query attention (MQA) in the two larger models.

Llama 2's training corpus includes a mix of data from publicly available sources, which Meta says does not include data from Meta's products or services. There were two trillion tokens of training data.

Meta used its Research Super Cluster and some internal production clusters for pre-training, with Nvidia A100 GPUs. Pre-training time ranged from 184K GPU-hours for the 7B-parameter model to 1.7M GPU-hours for the 70B-parameter model.

Fine-tuning Llama 2 Chat took months and involved both supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF). Meta used Ghost Attention (GAtt) to keep Llama 2 Chat from forgetting its system message (overall instruction) from turn to turn in a dialogue.

Is Llama 2 safe?

Some generative AI is notorious for making up (or hallucinating) answers, saying horrible things, and even recommending suicide. There are many situations where wrong answers can be dangerous, of course, and almost all LLMs come with boilerplate warnings. No matter how "safe" an AI is thought to be, you must always check its answers, since after all it is only a stochastic parrot.

Meta claims that Llama 2-chat is as safe or safer than other models, based on evaluation by human raters using ~2,000 adversarial prompts, as discussed in Meta's Llama 2 paper. Note Meta's caveat, however: There may be inherent bias of LLM evaluations due to limitations of the prompt set, subjectivity of the review guidelines, and subjectivity of individual raters. These safety evaluations are performed using content standards that are likely to be biased towards the Llama 2 Chat models.

Ethical considerations for Llama 2

In addition to the usual ethical and safety issues for LLMs, Llama 2 and Llama have an issue with one of the collections in their training corpus, the Books3 section of the Pile. In a class action lawsuit by Richard Kadrey, Sarah Silverman, and Christopher Golden, the plaintiffs allege that Meta has violated their copyrights by training on Books3, which includes their copyrighted books, and have asked for damages and restitution of profits. That lawsuit has not yet been litigated or settled.

At least one repository, the Eye, has recently responded to a DMCA takedown request from the Danish anti-piracy group Rights Alliance and removed Books3. Books3 is still available at other sites, although Rights Alliance is attempting to take them down as well. Ironically, Books3 was intended to democratize generative AI training, after OpenAI used private books datasets to train GPT-3.

Is Llama 2 open source?

In the introduction to my review of Llama 2 Chat and Code Llama, I described Llama 2 as "free almost-open-source." Why "almost" open source? It's because the Llama 2 license has a couple of restrictions. According to Stefano Maffulli of the OSI (Open Source Initiative),

Among other requirements, for a license to be Open Source, it may not discriminate against persons or groups or fields of endeavor (OSD points 5 and 6). Meta's license for the LLaMa models and code does not meet this standard; specifically, it puts restrictions on commercial use for some users (paragraph 2) and also restricts the use of the model and software for certain purposes (the Acceptable Use Policy).

Paragraph 2 of the Llama 2 community license agreement says

2. Additional Commercial Terms. If, on the Llama 2 version release date, the monthly active users of the products or services made available by or for Licensee, or Licensee's affiliates, is greater than 700 million monthly active users in the preceding calendar month, you must request a license from Meta, which Meta may grant to you in its sole discretion, and you are not authorized to exercise any of the rights under this Agreement unless or until Meta otherwise expressly grants you such rights.

That sounds like it has been designed to exclude AWS, Google Cloud, and Microsoft Azure. It follows the spirit of the Business Software License, originally developed by MariaDB. Most software developers don't really care about this kind of restriction, but open-source advocates care very much, indeed.

The Llama Acceptable Use Policy says, at a high level, that you can't use Llama to violate the law or others' rights; engage in, promote, incite, facilitate, or assist in the planning or development of activities that present a risk of death or bodily harm to individuals; intentionally deceive or mislead others; or fail to appropriately disclose to end users any known dangers of your AI system. They are saying that if you use Llama to develop weapons, create illegal drugs, create defamatory statements or images, or any of a long list of harms that have come out of other AI models, then you are violating your license.

I can sympathize with that attitude, and almost agree with it, but Mafulli is correct that it violates the Open Source Definition (OSD). I have to question, however, whether there's a clear a priori line between, say, using generative AI to design drugs that turn out to be legal and ones that turn out to be illegal.

For example, suppose you're designing a compound for pain relief using Llama, but post-market drug safety monitoring determines that it's highly addictive and it is subsequently classified as a Schedule 1 substance, and is banned. How do you know during the design process that you will be violating the acceptable use policy? I think that it's hard, if not impossible, to know in advance, and I doubt that the policy would hold up in court. Meanwhile, the OSI is trying to come up with a new definition of open source AI.

What is Code Llama?

Code Llama was trained by fine-tuning Llama 2 on code-specific datasets, specifically more of the same code that was used to train Llama 2 to begin with. Meta says that Code Llama can generate code, and natural language about code, from both code and natural language prompts, and can also be used for code completion and debugging. Code Llama supports many of the most popular languages being used today, including Python, C++, Java, PHP, TypeScript (JavaScript), C#, and Bash.

According to Meta AI:

Code Llama is a state-of-the-art LLM capable of generating code, and natural language about code, from both code and natural language prompts.

Code Llama is free for research and commercial use.

Code Llama is built on top of Llama 2 and is available in three models:

Code Llama, the foundational code model;

Code Llama - Python, specialized for Python;

and Code Llama - Instruct, which is fine-tuned for understanding natural language instructions.

In Meta's own benchmark testing, Code Llama outperformed state-of-the-art publicly available LLMs on code tasks.

There are three sizes of Code Llama with 7B, 13B, and 34B parameters. All the Code Llama models are trained on sequences of 16K tokens and show improvements on inputs with up to 100K tokens. The 7B and 13B base and instruct models have also been trained with fill-in-the-middle capabilities, which enables code completion.

The 7B model can be served from a single GPU. As you'd expect, the 34B model returns the best results, but the smaller models are faster and have lower latency, so they'd be more appropriate for use inside an editor doing code completion, which is why they are the models trained on fill-in-the-middle tasks.

Copyright © 2023 IDG Communications, Inc.

---