An introduction to large Language models, including vectorization, transformers, common nlp models, and considerations for designing apps with OpenAI tokens and PEFT.Large Language Models Utilizing B2B Software [https://baincapitalventures.com/insight/large-language-models-will-redefine-b2b-software/]IntroductionWhat is a Language Model?The probability distribution over strings of words is called the language Model. [1]For example, the knowledge that the word apple juice is more likely than apple juice concentrate is one simplest language models. This has replaced probabilistic methods nowadays with artificial neural networks. The dot product is used to measure how similar two words are in terms of character combination (syntax). But syntax can’t help us find the semantic (meaningful) relationship between two words. After all, the dot product will give us the result of zero because the character combinations are irrelevant. To solve this problem, we take vector representations (embedding) of words. Thus, we can train a model using these vector representations and obtain their context relations with each other.[1]Word2VecWord2Vec visual representation [https://www.tensorflow.org/tutorials/text/word2vec?hl=tr]The Word2Vec(Find Word Representation Vectors) method is a method that takes a word and establishes its relationships with the words on its right and left. Later, it determines the vector values accordingly and estimates the meanings of the words by converging on both sides of the words in the different texts that follow.[1]Thanks to the Word2Vec method, we can perform linear algebra operations between words[1]:king — man + woman ~= queenWord Vector Space Example [https://medium.com/opla/how-to-train-word-embeddings-using-small-datasets-9ced58b58fde]Artificial neural networks are the best-performing methods today and there are equations in these neural networks that map the input to the output, and these equations have parameters. First, we get an error during model training, during parameter adjustment we want the parameters to have the best appropriate values to reduce this mistake. To find the best values for these parameters, we employ the Gradient Descend technique.[1]In mathematics, we discover the optimum position by using the derivative. The derivative of the error is subtracted from the parameters at the end of each batch of Gradient Descend, and the parameters are brought closer to their target values.[1]Encoding and Decoding Layer Representation of an ANN Model [1]For example, you have ten items, the first of which is Apple and the second of which is Meta. Due to the effect of the parameters, these 10 elements are first reduced to 2 elements with the embedding layer, then these 2 elements are increased to 10 elements again, and since we know that apple and juice, meta and company should be matched, we provide the most appropriate parameters to be adjusted until they are fully matched, thanks to Gradient Descend. Finally, we pass the threshold value functions, such as softmax, and acquire the probability output. [1]Similarly, we provide the output and train it to match the input in a bidirectional manner (giving input from the output side), allowing us to group the words healthily thanks to the word vectors produced. Using the Pytorch module, we can easily code these deep-learning stages. Later, we employ a structure known as negative sampling since the method just described can be difficult to calculate for a variety of reasons, including the softmax function.[1]Bidirectional Model Representation [1]Instead of softmax, the sigmoid function is employed here, with the outcome being between 0 and 1. [1]Pre-trainingPre-training vs Fine-Tunning [https://www.researchgate.net/figure/BERT-Pre-training-vs-Fine-tuning-67_fig4_357046850]Pretraining is the first stage of language model training. The model is exposed to a huge corpus of unlabeled text input, such as books, papers, or web pages, during pretraining. The goal of pre-training is to teach the model the statistical patterns, language structures, and contextual correlations found in the training data. [1]Fine-tunningFine-tuning is the process of training a previously taught machine learning model on a new task or dataset to improve its performance or adapt it to a new domain. The conventional fine-tuning method involves freezing the pre-trained model’s initial layers and only changing the weights of the last few layers or adding task-specific layers on top. Training the model on a specific task or dataset teaches it to extract features relevant to the target problem and optimize its predictions accordingly. [1]Auto-encoder[https://towardsdatascience.com/applied-deep-learning-part-3-autoencoders-1c083af4d798]In our example, we employ an auto-encoder to reduce the size of our word vectors. An autoencoder is a form of artificial neural network that is mostly used for unsupervised learning tasks, such as dimensionality reduction and feature learning. It is intended to learn a compressed version of the input data by first encoding it into a lower-dimensional latent space and then decoding it back to the original input format. An autoencoder’s goal is to reconstruct the input as correctly as feasible. [1]An autoencoder’s architecture typically consists of two major components: an encoder and a decoder. The encoder converts input data to a lower-dimensional representation or code. This code’s dimensionality is frequently less than the dimensionality of the input data, forcing the autoencoder to learn a compressed form. The decoder then attempts to reconstruct the original input from the code. [1]LLM (Large Language Models)LLMs are sophisticated artificial intelligence (AI) models with millions of parameters that are designed to generate and understand human-like text. These models are trained on massive volumes of textual input and use deep learning techniques, notably transformers, to interpret and provide coherent and context-relevant replies. [1]LLMs are capable of a wide range of linguistic tasks, including language translation, text summarization, sentiment analysis, question answering, and even engaging in natural language dialogues. They can comprehend and generate human-like writing by understanding patterns, relationships, and contextual clues from the data on which they have been taught. [1]RNN[https://www.simplilearn.com/tutorials/deep-learning-tutorial/rnn]RNNs are neural networks that can handle sequential data by retaining information about previous inputs in an internal state or memory. However, because gradients might vanish or explode during training, ordinary RNNs often struggle to learn and keep long-range relationships.[1]LSTM[https://medium.com/deep-learning-turkiye/uzun-k%C4%B1sa-vadeli-bellek-lstm-b018c07174a3]Long Short-Term Memory (LSTM) is a form of recurrent neural network (RNN) architecture. LSTMs are intended to address the vanishing gradient problem and overcome the limitations of regular RNNs in capturing long-term dependencies.[1]LSTMs include a memory cell, which is essential for recording and retaining information over lengthy sequences. LSTMs can selectively forget or remember information based on the relevance and importance of each input thanks to the memory cell.[1]The LSTM cell is made up of numerous components such as input, output, and forget gates. These gates regulate the flow of data into and out of the memory cell, allowing the network to pick which data to keep and which to reject. The gates are activated by sigmoid activation functions, which allow them to create values ranging from 0 to 1, which define the extent of information flow.[1]ELMo[https://www.researchgate.net/figure/Google-ELMo-model-using-bidirectional-LSTM-model-for-information-flow-from-one-layer-to_fig1_359157231]ELMo (Embeddings from Language Models) is a 2018 deep contextualized word representation model. By employing a deep bidirectional language model, ELMo is designed to capture the contextual meaning of words in a phrase. [1]Unlike standard word embeddings like Word2Vec or GloVe, which assign fixed vectors to words regardless of context, ELMo builds dynamic word representations that change depending on the context. This is accomplished through the use of a bi-directional LSTM (Long Short-Term Memory) network, which processes the input sentence both forward and backward, capturing information from the complete context.[1]ELMo represents words as a layered representation of contextual information. The model’s layers capture several aspects of word meaning, ranging from low-level information (e.g., characters, subwords) to higher-level syntactic and semantic data. The ELMo may build context-sensitive word embeddings by merging these layers in a weighted manner.[1]Transformers[https://www.analyticsvidhya.com/blog/2019/06/understanding-transformers-nlp-state-of-the-art-models/]Transformers are a type of neural network design that was introduced in 2017 and has changed the field of natural language processing (NLP).[1]Transformers, as opposed to standard recurrent neural networks (RNNs), which process sequential data step by step, are based on the concept of a self-attention mechanism. By responding to diverse elements of the input sequence at the same time, this process allows the model to record relationships between words in a phrase.[1]Transformers have an encoder-decoder construction, but they can also be employed as an encoder-only instrument. Each transformer layer is divided into two sub-layers: a multi-head self-attention mechanism and a position-wise feed-forward neural network.[1]The self-attention mechanism allows the model to rank the value of different words in the input sequence based on their relationship to one another. It captures word dependencies and associations without the requirement for recurrent connections, making parallel processing more efficient. The position-wise feed-forward network individually applies non-linear changes to each location in the sequence.[1]Traditional sequential models offer various advantages over transformers. They can more efficiently handle long-term dependencies, capture the global context in a sentence, and parallelize computations.[1]Notably, the transformer design has been used in several well-known models, including the Transformer model, BERT (Bidirectional Encoder Representations from Transformers), GPT (Generative Pre-trained Transformer), and many others. By enabling more accurate and context-aware language understanding and creation, transformers have substantially enhanced the capabilities of NLP models.[1]Self-attention in Transformers[https://vaclavkosar.com/images/transformer-full-model.png]Self-attention is a method introduced in the transformer architecture for recording relationships and dependencies between different parts within a sequence, such as words in a sentence. It enables the model to concentrate on different parts of the input sequence to assess the significance or relevance of each element about others.[1]Each element in the sequence (for example, a word) is connected with three learned vectors in self-attention: a query vector, a key vector, and a value vector. These vectors are derived from the elements’ input embeddings. By assigning weights to each element depending on its similarity or relevance to the query, the self-attention mechanism computes a weighted total of the values.[1]A query’s and a key’s similarity or compatibility is computed as the dot product of their respective vectors, followed by a softmax operation to yield the weights. The output of the self-attention mechanism for a certain element is the weighted sum of the values multiplied by the weights.[1]Importantly, the model calculates self-attention in parallel for all items in the sequence, allowing it to capture global dependencies and linkages quickly. The self-attention weights show the value or attention attributed to each element depending on its relationship with others.[1]Self-attention allows the model to dynamically attend to different parts of the input sequence based on context and element dependencies. It has demonstrated effectiveness in capturing long-term dependencies and context information, resulting in significant gains in natural language processing tasks.[1]Popular NLP ModelsAfter 2018, the speed of NLP work began to accelerate. The models are shown below[1]:Evolutionary Tree [https://arxiv.org/abs/2304.13712]BERT[https://www.researchgate.net/figure/The-overall-structure-of-the-BERT-model_fig1_359301499]Google’s BERT (Bidirectional Encoder Representations from Transformers) language concept was launched in 2018. It is a neural network design with over 100 million parameters that is transformer-based. By understanding contextual links between words in a given sentence or text, BERT is supposed to understand and generate human-like language. [1]The bidirectional nature of BERT is an important feature. In contrast to prior models that merely analyzed a word’s left or right context, BERT employs a “masked language model” (MLM) pre-training target. BERT learns to predict masked words in a sentence based on the context provided by the surrounding words during the pre-training phase. This method enables BERT to capture bidirectional word dependencies and provide more contextually correct representations. [1]GPTOpenAI’s GPT (Generative Pre-trained Transformer) is a collection of massive language models. The GPT models are built on the ground-breaking transformer design.[1]OpenAI GPT-3, which was launched in 2020, is the most recent iteration and comprises a large neural network with 175 billion parameters. GPT-3 has been trained on a wide variety of online text data and is capable of producing human-like text in a variety of jobs. GPT models are unsupervised and pre-trained on a large corpus of text data. The model learns to anticipate the next word in a sentence based on the context of the preceding words during pre-training. The model can acquire complicated linguistic patterns, syntactic structures, and semantic links as a result of this process.[1]Following the completion of pre-training, GPT models can be fine-tuned on specific downstream tasks by providing labeled data for supervised learning. The process of fine-tuning adapts the model to perform effectively on tasks like text completion, question answering, summarization, and others. Because of their capacity to generate coherent and contextually relevant text, GPT models have received a lot of attention. Among other things, they can understand and respond to cues, carry on conversations, summarize text, translate languages, and even construct fictitious stories.[1]Hugging FaceHugging Face (also known as the Github of machine learning) is a company and open-source community focused on natural language processing (NLP) and deep learning. They are well-known for creating and maintaining the “Transformers” library, which has evolved into a prominent framework for interacting with cutting-edge NLP models.[1]Hugging Face Transformers provides a high-level API and pre-trained models that enable academics and developers to simply create and use powerful NLP models like BERT, GPT, Roberta, and many others. These models can be fine-tuned for specific downstream tasks or utilized out of the box for a variety of NLP-related activities such as text categorization, named entity identification, question answering, machine translation, and more.[1]🤗 TransformersHugging Face, in addition to the Transformers library, provides a variety of other open-source tools and resources for the NLP community. Among the notable offerings are[1]:Datasets: A library that gives users access to a variety of datasets for training and assessing NLP models.[1]Tokenizers: A tokenization library that allows for the efficient conversion of text data into tokens for model input.[1]Accelerate: A library that simplifies deep learning model training and inference by supporting distributed training and hardware acceleration.[1]Types of LLMsPopular pre-trained models. [1]All of these models can do additional skills as well; the ones listed here are based on their strongest points.[1]Base LLM“Base LLMs” are typically referred to as pre-trained language models that serve as a starting point for further customization or fine-tuning. These models have been pre-trained on massive volumes of text data and have a broad comprehension of language, syntax, and context. Unsupervised learning approaches are used to train base LLMs, which predict the next word in a sentence based on the preceding context.[1]Base LLMs are often adaptable and can be used for a wide range of natural language processing (NLP) activities like text generation, text completion, language translation, sentiment analysis, and more. They may not, however, be specifically fine-tuned for every given activity or sector. Base LLMs can be customized and fine-tuned by training them on unique datasets or providing task-specific prompts or instructions to adapt them to a given application. The model can become increasingly specialized and customized to the desired task or area as a result of this process.[1]T5[https://softos.tech/t5-the-groundbreaking-text-to-text-transfer-transformer-by-google/]T5 (Text-to-Text Transfer Transformer) is a flexible language model that may be modified for a variety of natural language processing (NLP) activities such as text classification, machine translation, summarization, and more. It employs a “text-to-text” paradigm in which both input and output are in the form of text. [1]To get better results, choose base models that have been trained on your domain or language. For example, for Turkish, consider the T5 model below:Turkish-NLP/t5-efficient-small-MLSUM-TR-fine-tuned · Hugging FaceThis model can be used with the HuggingFace API with only 3 lines of code:from transformers import T5ForConditionalGenerationmodel_name='t5-base'model = T5ForConditionalGeneration.from_pretrained(model_name)BERT (Bidirectional Encoder Representations from Transformers) is a transformer-based paradigm that excels at comprehending text context and meaning. It has a wide range of applications, including question answering, text categorization, named entity recognition, and sentiment analysis. BERT has been pre-trained on a huge corpus of text and can be tailored to specific needs. [1]from transformers import BertModelmodel_name='bert-base-uncased'model = BertModel.from_pretrained(model_name)Instruction- Tuned LLMLanguage models like GPT-3 (Generative Pre-trained Transformer 3) that may be fine-tuned with instructions or prompts to guide their output are examples of Instruction-Tuned LLMs. By supplying particular instructions or context, these models can generate text that corresponds to the desired conclusion. [1]GPT-3 is a cutting-edge language model that has been pre-trained on vast amounts of text data. It is well-known for producing coherent and contextually relevant responses. GPT-3 may be fine-tuned for specific instructions and excels in language translation, chatbot interactions, content development, and other tasks. [1]T5 vs GPTArchitectureT5 uses a “text-to-text” framework and is based on the transformer architecture. It is composed of an encoder-decoder structure, with both input and output in the form of text. [1]GPT-3 is based on the transformer architecture as well, however, it employs a “decoder-only” approach. It generates text by guessing the next word in a sequence depending on the context that came before it. The ability of GPT-3 to generate coherent and contextually relevant replies is well recognized.Use CasesT5 is a versatile model that can be modified for a variety of NLP tasks such as text categorization, language translation, summarization, question answering, and others. Its “text-to-text” structure enables quick adaptability to new jobs. [1]GPT-3 is most recognized for its text-generating features. It can be used to generate content, engage with chatbots, tell stories, and write creatively. Based on the context, GPT-3 excels at producing human-like prose. [1]Model Size and ScaleT5 is available in several sizes, the largest of which has 11 billion parameters. [1]GPT-3 is an exceptionally big model, the largest in the GPT series at the moment, with 175 billion parameters. Because of its huge size, it can capture nuanced linguistic patterns and provide detailed, contextually rich replies. [1]Output ConsistencyT5’s text-to-text foundation and fine-tuning procedure provide you with more control and consistency in your outputs. T5 can be trained on specific datasets or fine-tuned for specific tasks, allowing the model’s responses to be more aligned with the desired goal. T5 can produce more consistent results across varied inputs by offering clear instructions and prompts.[1]The consistency of outcomes may also be affected by the quality and diversity of training data, the fine-tuning procedure, and the task’s specific needs. To maintain acceptable levels of consistency, it is recommended to experiment, iterate, and evaluate the performance of the models in the given environment.[1]Things to Consider When Developing Apps That Use OpenAI TokenCounting Tokens of a ChatGPT Request [https://blog.devgenius.io/counting-tokens-for-openai-gpt-3-api-59c8e0812eeb]Prompt OptimizationWhen you perform a project with an OpenAI token, you pay per token in input and output (think of token count as word count), thus it is more profitable to keep your prompts brief and simple and to run your tasks with the fewest prompts possible. [1]OpenAI message token = 4 bytesEvery character ~=2 bytesEvery word ~=4 bytes or 1 OpenAI message tokenEvery emoji = 4 bytesWhen developing a prompt, there are 3 basic components. These are: “output”, “output format”, and “details of the output”. [1]Because LLMs transform input to vectors, you can get a similar result by typing as few characters as feasible, independent of grammar. As an example, [1]:“Can you create me a rhyming poem with two lines on butterflies?”Here “2 lines” indicates the output format. “Write a poem” indicates the output and the “rhyming, and on butterflies” part indicates the details of the output. [1]We can also write it with minimum characters like this[1]:“2 lines 🦋 poem”In response, instead of repeating the same things for a long time, we only say, reply “OK” to me when you understand. So the answers are optimized like this[1]:Hello! Of course, I would be happy to assist you. I look forward to helping you. You can start explaining.Instead answers like this[1]:OKEmojis can also be used for output too so we pay less for output. We can also pay attention to this by specifying the output limit as a prompt parameter.[1]You must send the entire history when using OpenAI tokens because OpenAI tokens are stateless, they do not keep track of your session. As a result, your token number grows rapidly. The solution is summarizing, which reduces the length of the history by taking a summary of the dialogue at regular intervals.[1]Prompt ParametersChatGPT Playground Parameters [2]Preset/ModeHere OpenAI provides a number of ready-to-be-used presets for different uses of GPT-3.[2]Preset drop-down bar [2]Select the “English to French” preset. When you choose a preset, the contents of the text area are updated with a predefined training text for it. The settings in the right sidebar are also updated. [2]ModelThere are different models that Openai serves. They all have different charges per token and performance and accuracy. GPT-3.5-Turbo is very fast. Ada is also fast and cheapest whereas Davinci-003 is the most expensive one with the highest accuracy. [2]Maximum lengthThe output length. You also need to pay by the output. The default response length is 256 characters. [2]TemperatureOne of the most important settings to control the output of the GPT-3 engine is the temperature. This setting controls the randomness of the generated text. A value of 0 makes the engine deterministic, which means that it will always generate the same output for a given input text. A value of 1 makes the engine take the most risks and use a lot of creativity. I like to start prototyping an application by setting the temperature to 0, so let’s start by doing that. The “Top P” parameter that appears below the temperature also has some control over the randomness of the response, so make sure that it is at its default value of 1. Leave all other parameters also at their default values. [2]Top PIt is an alternative way of controlling the randomness and creativity of the text generated by GPT-3. The OpenAI documentation recommends that only one of Temperature and Top P are used, so when using one of them, make sure that the other is set to 1. [2]The author says he got the best results with this setting in his limited experiment [2]RolesThere are 3 roles: These are system role, assistant role, and user role.[1]System RoleThis is the point at which we want the responsive language model to reply with the type of character.[1]Assistant RoleGPT-3’s answers are shown here.[1]User RoleUser messages are sent here. Be aware that GPT-3 does not like input strings that end in a space, as this causes weird and sometimes unpredictable behaviors. [1][2]Prompt InjectionLet’s say you built your application to use ChatGPT-3, and you have restricted the API control. Consider the following scenario: You save the response from this page as text in your database. A malicious (or inadvertent) user is making requests outside of the area you have narrowed with a prompt injection here and using it in various ways. This will both disrupt the working principles of your application and cause problems, as well as cause your token to be consumed against your will. [1]Here’s a cheap API called OpenAI Moderation API that catches those who fall outside of societal norms[1]:OpenAI APIApart from that, we can also make our own domain-moderation AI using ChatGPT-3 to prevent going out of our domain[1]:const MODERATION_PROMPT = [ { "role":"system", "content":"You are a text classifier that detects if text is only about coffee, and return 'True' else return 'False'. *NEVER* write something different than 'True' or 'False'. Reply 'OK' if you understand.",}, { "role": "assistant", "content": "OK", },];if (getAIResponse(moderation).contains("False")){ print("AI Moderation failed for:", limitedQuery); return failedModeration();}PALM2[https://arstechnica.com/information-technology/2023/05/googles-top-ai-model-palm-2-hopes-to-upstage-gpt-4-in-generative-mastery/]Pathways Language Model 2, or PaLM 2, is Google’s next-generation large language model (LLM). It is a neural network that has been trained on a vast dataset of text and code and can generate text, translate languages, write various types of creative material, and provide relevant answers to your questions. Although the PaLM 2 is still in progress, it has already trained to do a variety of tasks, including [3]:Breaking down a big task into smaller subtasks, understanding the subtleties of human language, understanding idioms and riddles, more than 100 languages are translated, writing various types of creative stuff, such as poems, code, scripts, musical pieces, emails, letters, and so on; educationally answering your questions, even if they are open-ended, difficult, or unusual. [3]The PaLM 2 is based on Google’s history of groundbreaking research in machine learning and responsible AI. It is a versatile tool that may be used for several tasks, including [3]:Text creation, language translation, creating several types of creative stuff, and answering your queries in an informative way. assisting you in learning new things, and becoming more creative and productive. [3]The following conclusions were reached while creating PALM2: If we clean the data utilized by the model, the model’s success rate will skyrocket. The quantity of data is less significant than its quality. In other words, the best strategy to gain greater success with smaller models is to train using better-cleaned, higher-quality data; this method is far superior to modifying the model’s pre-tune and fine-tuning it. [1]However, there is a cost. Bigger models require more tokens if you want to make them bigger. The quality begins to deteriorate as the number of tokens increases. PALM2-Small outperformed PALM2-Medium and PALM2-Large in learning text and answering questions in several languages. Large models, on the other hand, perform better when the question is not in the text because they can identify and extract the answer from the training data. [1]Encoders and decoders are components of models. The encoder part understands the given input, and the decoder produces output based on the meaning. Although most of the models can interpret in Turkish at the encoding stage, the decoding phase, that is, the process of producing Turkish answers, is significantly more challenging.[1]How do we set the weights of a model?[1]https://arxiv.org/abs/2304.09151What kinds of data can I use with languages that have very few sources?[1]Bilex Rx: Lexical Data Augmentation for Massively Multilingual Machine TranslationCan we predict the language’s accuracy with a small amount of data(big models tuned in hyperparameter in small levels later the overall success is predicted)?[1]Scaling Laws for Multilingual Neural Machine TranslationLLMs can perform tasks it has not seen or it has seen in another language with a small bit of quality data [1]:The unreasonable effectiveness of few-shot learning for machine translationIn interlingual translation, including that language’s culture and carrying out the process in that manner, that is, performing more than direct translation [1]:https://arxiv.org/abs/2301.10309Parameter Efficient Fine Tunning (PEFT)[https://www.youtube.com/watch?v=FmcCICoY7eU]Full-fine tunningFull model fine-tuning is also known as full parameter fine-tuning.[1]How big should a language model be to be considered a Large Language Model?A large language model (LLM) does not have an established definition. Most experts, however, believe that an LLM should have at least 100 billion parameters. This is because models with fewer parameters cannot learn the intricate links between words and sentences required for tasks such as natural language processing and creation.[1]PALM2 contains 540 billion, GPT-3 has 175 billion, T5 has 11 billion, and BERT has 110 million parameters. These models are attempting to create multi-language, multi-task models.[1]In Deep learning models, artificial neural network’s 7th and 8th layers/middle levels have been seen to collect semantic information. Each layer distributes duties independently. While one layer is focused on one thing, the other is focused on something different. [1]The model consumes far more memory than simply placing the model on the GPU. The space needed [1]:The Components on GPU Memory = Model weights * 4 bytes + Optimizer states * 2–8 bytes + Gradients * 4 bytes + Forward activations (sequence length + hidden size+ batch size) + Temporary buffersAs a result, a 7 billion parameter model uses around 70 gigabytes of GPU. [1]PEFT(Probabilistic Evolutionary Hyperparameter Tuning)When using a model LLM on a mission basis, we must deploy it ten times; instead, PEFT can be used. Because PEFT models fine-tune specific sections of the model as an add-on, we can deploy one large model and nine mini models. PEFT also aids in the optimization of hyperparameters. [1]PEFT uses a probabilistic strategy to determine hyperparameter values, allowing it to examine a wide range of hyperparameters. Using evolutionary algorithms to optimize hyperparameters and uncover the optimum combinations. It allows automatic tuning, removing the need for users to manually tune hyperparameters. It accelerates the hyperparameter optimization process and allows for more extended study by offering parallel processing assistance. [1]PEFT TypesIn essence, three basic approaches have been observed to be more effective in recent years: additive (add new parameters to the transformer), selective (manually selected parameters that can be trained on the model), and reparametrization (we express and update the parameters in low-rank). Adapters employ an additive approach.[1]PEFT Types [https://www.arxiv-vanity.com/papers/2303.15647/]Additive MethodTransformers receive new trainable layers. Other layers cannot be trained. We can add up to 1% of the total model. Bottleneck architecture is favored here to use as little data as feasible. Bottleneck architecture reduces the original dimension to a smaller scale and then maps it back to the original dimension.[1]Selective MethodThe parameters derived from Bayes are trained (frozen). It investigates the use of a formula and which parameters in the model should be learned using this formula. [1]Reparametrization-based / Low-rank MethodWhen the dimension is reduced in the low-rank method, which is widely used in artificial intelligence, both the processing speed and the generated information rise. LoRa is a well-known Low-rank algorithm. [1]The LoRA is applied at the Feed-forward state of a Transformer [https://xiaosean5408.medium.com/fine-tuning-llms-made-easy-with-lora-and-generative-ai-stable-diffusion-lora-39ff27480fda]PEFT is commonly utilized with 2 libraries: Transformer-adapters and HuggingFace PEFT. [1]from transformers import BertModelWithHeadsmodel = BertModelWithHeads.from_pretrained(model_path)model.add_adapter('imdb_sentiment')model.add_classification_head('imdb_sentiment', num_labels=2)model.train_adapter('imdb_sentiment')In the above example, while vanilla training takes 8 minutes, it takes 1 minute with the adapter.[1]In a BERT-based model with 105.83M parameters, 1.42M parameters are trainable, which shows an efficiency increase of 1.34%.[1]PEFT-Lora Example Implementationfrom peft import get_peft_config, get_peft_model, get_peft_model_state_dict, LoraConfig, TaskTypefrom transformers import AutoConfig, AutoModelForSeq2SeqLMimport torchpeft_config = LoraConfig(task_type=TaskType.SEQ_2_SEQ_LM, inference_mode=False, r=8, lora_alpha=32,lora_dropout=0.1)model = AutoModelForSeq2SeqLM.from_pretrained(model_name_or_path, load_in_8bit=True, torch_dtype=torch.float16, device_map='auto')model = get_peft_model(model, peft_config)model.print_trainable_parameters()Trainable parameters: 294k, All parameters: 60M, Trainable ratio: 0.48%Inference EfficiencyThe speed and computational efficiency with which a machine learning model makes predictions or inferences is referred to as inference efficiency. It focuses on optimizing the inference process to make faster and more accurate predictions, particularly in real-time or resource-constrained contexts. Model compression, quantization, and hardware acceleration are used to improve inference efficiency without compromising model performance.[1]Few-shot learningFew-shot learning is a machine learning subfield that addresses the problem of learning from a small number of labeled samples. Traditional machine learning methods require a huge quantity of labeled data for training, while the goal of few-shot learning is to create models that can learn from a small number of examples. This entails using prior knowledge, transfer learning, or meta-learning approaches to generalize from limited data and generate accurate predictions on previously unseen samples.[1]Contrastive LearningContrastive learning is a self-supervised learning technique that uses unlabeled data to learn usable representations. It entails training a model to distinguish between positive and negative data pairings. Contrastive learning helps the model to capture important and discriminative features by bringing similar samples closer together and pushing dissimilar samples apart in the learned representation space. This strategy has proven to be effective in a variety of fields, including computer vision and natural language processing.[1]Domain AdaptionDomain adaptation is the process of moving knowledge from one domain to another, where the distribution of facts may change. Because of the distribution change, it is frequently difficult to apply models learned on one domain straight to another. By aligning their feature representations, domain adaptation strategies strive to reduce the disparity between the source and target domains. This allows models to generalize better and perform better on the target domain, even when labeled data in the target domain is scarce.[1]References[1]Matematik ve Yapay Zeka Enstitüsü, (20 May 2023), Büyük Dil Modelleri Günü:[https://www.youtube.com/watch?v=I-aCXJqPzYs][2] Miguel Grinberg, (25 August 2020), Ultimate Guide OpenAI GPT 3 Language Model:[https://www.twilio.com/blog/ultimate-guide-openai-gpt-3-language-model][3] Output from Bard, Google to cbarkinozer, 26 May 2023. The output was generated in response to the prompt, ‘What is PALM2?’.An Overview of Large Language Models was originally published in Better Programming on Medium, where people are continuing the conversation by highlighting and responding to this story.