[{"data":1,"prerenderedAt":704},["ShallowReactive",2],{"/en-us/blog/learning-python-with-a-little-help-from-ai-code-suggestions/":3,"navigation-en-us":37,"banner-en-us":454,"footer-en-us":466,"Michael Friedrich":676,"next-steps-en-us":689},{"_path":4,"_dir":5,"_draft":6,"_partial":6,"_locale":7,"seo":8,"content":16,"config":27,"_id":30,"_type":31,"title":32,"_source":33,"_file":34,"_stem":35,"_extension":36},"/en-us/blog/learning-python-with-a-little-help-from-ai-code-suggestions","blog",false,"",{"title":9,"description":10,"ogTitle":9,"ogDescription":10,"noIndex":6,"ogImage":11,"ogUrl":12,"ogSiteName":13,"ogType":14,"canonicalUrls":12,"schema":15},"Learning Python with a little help from AI","Use this guided tutorial, along with GitLab Duo Code Suggestions, to learn a new programming language.","https://res.cloudinary.com/about-gitlab-com/image/upload/v1749663918/Blog/Hero%20Images/aipower.jpg","https://about.gitlab.com/blog/learning-python-with-a-little-help-from-ai-code-suggestions","https://about.gitlab.com","article","\n                        {\n        \"@context\": \"https://schema.org\",\n        \"@type\": \"Article\",\n        \"headline\": \"Learning Python with a little help from AI\",\n        \"author\": [{\"@type\":\"Person\",\"name\":\"Michael Friedrich\"}],\n        \"datePublished\": \"2023-11-09\",\n      }",{"title":9,"description":10,"authors":17,"heroImage":11,"date":19,"body":20,"category":21,"tags":22},[18],"Michael Friedrich","2023-11-09","\nLearning a new programming language can help broaden your software development expertise, open career opportunities, or create fun challenges. However, it can be difficult to decide on one specific approach to learning a new language. Artificial intelligence (AI) can help. In this tutorial, you'll learn how to leverage AI-powered GitLab Duo Code Suggestions for a guided experience in learning the Python programming language with a practical hands-on example.\n\n- [Preparations](#preparations)\n  - [VS Code](#vs-code)\n  - [Code Suggestions](#code-suggestions)\n- [Learning a new programming language: Python](#learning-a-new-programming-language-python)\n    - [Development environment for Python](#development-environment-for-python)\n    - [Hello, World](#hello-world)\n- [Start learning Python with a practical example](#start-learning-python-with-a-practical-example)\n    - [Define variables and print them](#define-variables-and-print-them)\n    - [Explore variable types](#explore-variable-types)\n- [File I/O: Read and print a log file](#file-io-read-and-print-a-log-file)\n- [Flow control](#flow-control)\n    - [Loops and lists to collect files](#loops-and-lists-to-collect-files)\n    - [Conditionally collect files](#conditionally-collect-files)\n- [Functions](#functions)\n    - [Start with a simple log format](#start-with-a-simple-log-format)\n    - [String and data structure operations](#string-and-data-structure-operations)\n    - [Parse log files using regular expressions](#parse-log-files-using-regular-expressions)\n    - [Advanced log format: auth.log](#advanced-log-format-authlog)\n    - [Parsing more types: Structured logging](#parsing-more-types-structured-logging)\n- [Printing results and formatting](#printing-results-and-formatting)\n- [Dependency management and continuous verification](#dependency-management-and-continuous-verification)\n    - [Pip and pyenv: Bringing structure into Python](#pip-and-pyenv-bringing-structure-into-python)\n    - [Automation: Configure CI/CD pipeline for Python](#automation-configure-cicd-pipeline-for-python)\n- [What is next](#what-is-next)\n    - [Async learning exercises](#async-learning-exercises)\n    - [Share your feedback](#share-your-feedback)\n\n## Preparations \n\nChoose your [preferred and supported IDE](https://docs.gitlab.com/ee/user/project/repository/code_suggestions.html#enable-code-suggestions-in-other-ides-and-editors), and follow the documentation to enable Code Suggestions for [GitLab.com SaaS](https://docs.gitlab.com/ee/user/project/repository/code_suggestions.html#enable-code-suggestions-on-gitlab-saas) or [GitLab self-managed instances](https://docs.gitlab.com/ee/user/project/repository/code_suggestions.html#enable-code-suggestions-on-self-managed-gitlab).\n\nProgramming languages can require installing the language interpreter command-line tools or compilers that generate binaries from source code to build and run the application.\n\n**Tip:** You can also use [GitLab Remote Development workspaces](/blog/quick-start-guide-for-gitlab-workspaces/) to create your own cloud development environments, instead of local development environments. This blog post focuses on using VS Code and the GitLab Web IDE. \n\n### VS Code\n\n[Install VS Code](https://code.visualstudio.com/download) on your client, and open it. Navigate to the `Extensions` menu and search for `gitlab workflow`. Install the [GitLab Workflow extension for VS Code](https://marketplace.visualstudio.com/items?itemName=GitLab.gitlab-workflow). VS Code will also detect the programming languages, and offer to install additional plugins for syntax highlighting and development experience. For example, install the [Python extension](https://marketplace.visualstudio.com/items?itemName=ms-python.python).\n\n### Code Suggestions\n\nFamiliarize yourself with suggestions before actually verifying the suggestions. GitLab Duo Code Suggestions are provided as you type, so you do not need use specific keyboard shortcuts. To accept a code suggestion, press the `tab` key. Also note that writing new code works more reliably than refactoring existing code. AI is non-deterministic, which means that the same suggestion may not be repeated after deleting the code suggestion. While Code Suggestions is in Beta, we are working on improving the accuracy of generated content overall. Please review the [known limitations](https://docs.gitlab.com/ee/user/project/repository/code_suggestions.html#known-limitations), as this could affect your learning experience.\n\n**Tip:** The latest release of Code Suggestions supports multiline instructions. You can refine the specifications to your needs to get better suggestions. We will practice this method throughout the blog post.\n\n## Learning a new programming language: Python  \n\nNow, let's dig into learning Python, which is one of the [supported languages in Code Suggestions](https://docs.gitlab.com/ee/user/project/repository/code_suggestions.html#supported-languages). \n\nBefore diving into the source code, make sure to set up your development environment.\n\n### Development environment for Python \n\n1) Create a new project `learn-python-ai` in GitLab, and clone the project into your development environment. All code snippets are available in this [\"Learn Python with AI\" project](https://gitlab.com/gitlab-de/use-cases/ai/learn-with-ai/learn-python-ai).\n\n```shell\ngit clone https://gitlab.com/NAMESPACE/learn-python-ai.git\n\ncd learn-python-ai\n\ngit status\n```\n\n2) Install Python and the build toolchain. Example on macOS using Homebrew:\n\n```\nbrew install python\n```\n\n3) Consider adding a `.gitignore` file for Python, for example this [.gitignore template for Python](https://gitlab.com/gitlab-org/gitlab/-/blob/master/vendor/gitignore/Python.gitignore?ref_type=heads). \n\nYou are all set to learn Python! \n\n### Hello, World\n\nStart your learning journey in the [official documentation](https://www.python.org/about/gettingstarted/), and review the linked resources, for example, the [Python tutorial](https://docs.python.org/3/tutorial/index.html). The [library](https://docs.python.org/3/library/index.html) and [language reference](https://docs.python.org/3/reference/index.html) documentation can be helpful, too. \n\n**Tip:** When I touched base with Python in 2005, I did not have many use cases except as a framework to test Windows 2000 drivers. Later, in 2016, I refreshed my knowledge with the book \"Head First Python, 2nd Edition,\" providing great practical examples for the best learning experience – two weeks later, I could explain the differences between Python 2 and 3. You do not need to worry about Python 2 – it has been deprecated some years ago, and we will focus only on Python 3 in this blog post. In August 2023, \"[Head First Python, 3rd Edition](https://www.oreilly.com/library/view/head-first-python/9781492051282/)\" was published. The book provides a great learning resource, along with the exercises shared in this blog post. \n\nCreate a new file `hello.py` in the root directory of the project and start with a comment saying `# Hello world`. Review and accept the suggestion by pressing the `tab` key and save the file (keyboard shortcut: cmd s). \n\n```\n# Hello world\n```\n\nCommit the change to the Git repository. In VS Code, use the keyboard shortcut `ctrl shift G`, add a commit message, and hit `cmd enter` to submit. \n\nUse the command palette (`cmd shift p`) and search for `create terminal` to open a new terminal. Run the code with the Python interpreter. On macOS, the binary from Homebrew is called `python3`, other operating systems and distributions might use `python` without the version.\n\n```shell\npython3 hello.py\n```\n\n![Hello World, hello GitLab Duo Code Suggestions](https://about.gitlab.com/images/blogimages/learn-python-with-ai-code-suggestions-getting-started/vs_code_python_code_sugestions_hello_world.png)\n\n**Tip:** Adding code comments in Python starting with the `#` character before you start writing a function or algorithm will help Code Suggestions with more context to provide better suggestions. In the example above, we did that with `# Hello world`, and will continue doing so in the next exercises.\n\nAdd `hello.py` to Git, commit all changes and push them to your GitLab project.\n\n```shell\ngit add hello.py\n\ngit commit -avm \"Initialize Python\"\n\ngit push\n```\n\nThe source code for all exercises in this blog post is available in this [\"Learn Python with AI\" project](https://gitlab.com/gitlab-de/use-cases/ai/learn-with-ai/learn-python-ai).\n\n## Start learning Python with a practical example \n\nThe learning goal in the following sections involves diving into the language datatypes, variables, flow control, and functions. We will also look into file operations, string parsing, and data structure operations for printing the results. The exercises will help build a command-line application that reads different log formats, works with the data, and provides a summary. This will be the foundation for future projects that fetch logs from REST APIs, and inspire more ideas such as rendering images, creating a web server, or adding Observability metrics.\n\n![Parsing log files into structured objects, example result after following the exercises](https://about.gitlab.com/images/blogimages/learn-python-with-ai-code-suggestions-getting-started/vs_code_terminal_parsing_logs_and_pretty_print_results.png)\n\nAs an experienced admin, you can put the script into production and use real-world log format exmples. Parsing and analyzing logs in stressful production incidents can be time-consuming. A local CLI tool is sometimes faster than a log management tool.\n\nLet's get started: Create a new file called `log_reader.py` in the directory root, add it to Git, and create a Git commit.\n\n### Define variables and print them\n\nAs a first step, we need to define the log files location, and the expected file suffix. Therefore, let's create two variables and print them. Actually, ask Code Suggestions to do that for you by writing only the code comments and accepting the suggestions. Sometimes, you need to experiment with suggestions and delete already accepted code blocks. Do not worry – the quality of the suggestions will improve over time as the model generates better suggestions with more context.\n\n![Define log path and file suffix variables](https://about.gitlab.com/images/blogimages/learn-python-with-ai-code-suggestions-getting-started/vs_code_code_suggestions_log_reader_variables_01.png){: .shadow}\n\n![Print the variables to verify](https://about.gitlab.com/images/blogimages/learn-python-with-ai-code-suggestions-getting-started/vs_code_code_suggestions_log_reader_variables_02.png){: .shadow}\n\n```python\n# Specify the path and file suffix in variables\npath = '/var/log/'\nfile_suffix = '.log'\n\n# Print the variables \n\nprint(path)\nprint(file_suffix)\n```\n\nNavigate into the VS Code terminal and run the Python script:\n\n```shell\npython3 log_reader.py\n```\n\n![VS Code terminal, printing the variables](https://about.gitlab.com/images/blogimages/learn-python-with-ai-code-suggestions-getting-started/vs_code_terminal_print_variables.png)\n\nPython supports many different types in the [standard library](https://docs.python.org/3/library/index.html). Most common types are: Numeric (int, float, complex), Boolean (True, False), and String (str). Data structures include support for lists, tuples, and dictionaries. \n\n### Explore variable types \n\nTo practice different variable types, let's define a limit of log files to read as a variable with the `integer` type.\n\n![Log file variable](https://about.gitlab.com/images/blogimages/learn-python-with-ai-code-suggestions-getting-started/vs_code_code_suggestions_log_reader_variables_03.png){: .shadow}\n\n```python\n# Define log file limit variable \nlog_file_limit = 1024 \n```\n\nCreate a Boolean variable that forces to read all files in the directory, no matter the log file suffix. \n\n```python\n# Define boolean variable whether to read all files recursively\nread_all_files_recursively = True\n```\n\n## File I/O: Read and print a log file\n\nCreate a directory called `log-data` in your project tree. You can copy all file examples from the [log-data directory in the example project](https://gitlab.com/gitlab-de/use-cases/ai/learn-with-ai/learn-python-ai/-/tree/main/log-data?ref_type=heads).  \n\nCreate a new file `sample.log` with the following content, or any other two lines that provide a different message at the end.\n\n```\nOct 17 00:00:04 ebpf-chaos systemd[1]: dpkg-db-backup.service: Deactivated successfully.\nOct 17 00:00:04 ebpf-chaos systemd[1]: Finished Daily dpkg database backup service.\n```\n\nInstruct Code Suggestions to read the file `log-data/sample.log` and print the content. \n\n![Code Suggestions: Read log file and print it](https://about.gitlab.com/images/blogimages/learn-python-with-ai-code-suggestions-getting-started/vs_code_code_suggestions_read_log_file_and_print.png){: .shadow}\n\n```python\n# Read the file in log-data/sample.log and print its content\nwith open('log-data/sample.log', 'r') as f:\n    print(f.read())\n```\n\n**Tip:** You will notice the indent here. The `with open() as f:` statement opens a new scope where `f` is available as stream. This flow requires indenting )`tab`) the code block, and perform actions in this scope, calling `f.read()` to read the file contents, and passing the immediate value as parameter into the `print()` function.\n\nNavigate into the terminal, and run the script again with `python3 log_reader.py`. You will see the file content shown in the VS Code editor, also printed into the terminal.\n\n![VS Code terminal: Read log file, and print it](https://about.gitlab.com/images/blogimages/learn-python-with-ai-code-suggestions-getting-started/vs_code_terminal_print_logfile_show_file_sample.png)\n\n## Flow control \n\nReading one log file is not enough – we want to analyze all files in a given directory recursively. For the next exercise, we instruct Code Suggestions to create an index of all files. \n\nPrepare the `log-data` directory with more example files from the [log-data directory in the example project](https://gitlab.com/gitlab-de/use-cases/ai/learn-with-ai/learn-python-ai/-/tree/main/log-data?ref_type=heads). The directory tree should look as follows:\n\n```shell\ntree log-data                                                             ─╯\nlog-data\n├── sample.log\n└── var\n    └── log\n        ├── auth.log\n        ├── syslog.log\n        └── syslog_structured.log\n\n3 directories, 4 files\n```\n\n### Loops and lists to collect files \n\nModify the `path` variable to use the value `log-data/`. \n\n```python\n# Specify the path and file suffix in variables\npath = 'log-data/'\nfile_suffix = '.log'\n```\n\nTell Code Suggestions to read all file paths in the directory into a list. After the collection loop, print the list of file paths. \n\n```python\n# Read all file paths in the directory into a list\n\n# Print the list of log file paths\n```\n\n![Code Suggestion, collect file paths](https://about.gitlab.com/images/blogimages/learn-python-with-ai-code-suggestions-getting-started/vs_code_code_suggestions_collect_files.png){: .shadow}\n\nAn example of a suggestion could look like this:\n\n```python\n# Read all file paths in the directory into a list\nimport os\n\n# Read all file paths in the directory into a list\nlog_files = []\nfor root, directories, files in os.walk(path):\n    for file in files:\n        if file.endswith(file_suffix):\n            log_files.append(os.path.join(root, file))\n\n# Print the list of log file paths\n\nprint(log_files)\n```\n\nLet's analyze what happens here: The `import os` statement is required to bring the `os` library into the current scope, and we are able to call `os.walk()` later. The `log_files` [list](https://docs.python.org/3/library/stdtypes.html#sequence-types-list-tuple-range) is initialized as an empty list using empty brackets. `os.walk()` expects a path as directory tree to start searching for files. The function returns a generator object, which will walk the directory tree recursively. The returned tuples include the collected files, which we want to access in a loop iteraor, using the `for` loop notation. When the file ends with the defined suffix, its full path is appended to the list. This is an optimized and complex example. Do not worry about understanding everything just yet - we will revisit this later. \n\n### Conditionally collect files \n\nModify the instructions and add the Boolean flag evaluation for `read_all_files_recursively` as an input to the glob() function. \n\n```python\n# Read all file paths in the top level directory into a list\n# If read_all_files_recursively is True, read all files recursively\n```\n\n![Code Suggestions, read files recursively conditionally](https://about.gitlab.com/images/blogimages/learn-python-with-ai-code-suggestions-getting-started/vs_code_code_suggestions_read_files_recursively_conditionally.png){: .shadow}\n\n```\n# Read all file paths in the top level directory \nimport os\n\n# Read all file paths in the top level directory into a list\n# If read_all_files_recursively is True, read all files recursively\n\nlog_files = []\nfor file in os.listdir(path):\n    if file.endswith(file_suffix):\n        log_files.append(os.path.join(path, file))\n\nif read_all_files_recursively:\n    for root, directories, files in os.walk(path):\n        for file in files:\n            if file.endswith(file_suffix):               \n                log_files.append(os.path.join(root, file))\n\n# Print log_files\nprint(log_files)\n```\n\nThe result is not optimal yet because it always executes the first loop, and optionally the second loop. This flow leads to duplicated results when the script is executed.\n\n```\npython3 log_reader.py\n\n['log-data/sample.log', 'log-data/sample.log', 'log-data/var/log/auth.log']\n```\n\nExperiment with Code Suggestions instructions to get a solution for the problem. There are different approaches you can take: \n\n1) A potential solution is to wrap the source code into an if-then-else block, and move the `os.listdir()` loop into the else-block. \n\n```python\nif read_all_files_recursively:\n    for root, directories, files in os.walk(path):\n        for file in files:\n            if file.endswith(file_suffix):               \n                log_files.append(os.path.join(root, file))\nelse:\n    for file in os.listdir(path):\n        if file.endswith(file_suffix):\n            log_files.append(os.path.join(path, file))  \n\n```\n\n2) Alternatively, do not use `append()` to always add a new list entry, but check if the item exists in the list first. \n\n```python\nfor file in os.listdir(path):\n    if file.endswith(file_suffix):\n        # check if the entry exists in the list already\n        if os.path.isfile(os.path.join(path, file)):\n            log_files.append(os.path.join(path, file))\n\nif read_all_files_recursively:\n    for root, directories, files in os.walk(path):\n        for file in files:\n            if file.endswith(file_suffix):\n                # check if the entry exists in the list already\n                if file not in log_files:\n                    log_files.append(os.path.join(root, file))\n```\n\n3) Or, we could eliminate duplicate entries after collecting all items. Python allows converting lists into [sets](https://docs.python.org/3/library/stdtypes.html#set-types-set-frozenset), which hold unique entries. After applying `set()`, you can again convert the set back into a list. Code Suggestions knows about this possibility, and will help with the comment `# Ensure that only unique file paths are in the list` \n\n![Code Suggestions, converting a list to unique items](https://about.gitlab.com/images/blogimages/learn-python-with-ai-code-suggestions-getting-started/vs_code_code_suggestions_unique_list.png){: .shadow}\n\n```python\n# Ensure that only unique file paths are in the list\n\nlog_files = list(set(log_files))\n```\n\n4) Take a step back and evaluate whether the variable read_all_files_recursively makes sense. Maybe the default behavior should just be reading all files recursively?\n\n**Tip for testing different paths in VS Code:** Select the code blocks, and press [`cmd /` on macOS](https://code.visualstudio.com/docs/getstarted/keybindings) to comment out the code. \n\n## Functions \n\nLet's create a function called `parse_log_file` that parses a log file, and returns the extracted data. We will define the expected log format and columns to extract, following the [syslog format specification](https://en.wikipedia.org/wiki/Syslog). There are different log format types and also customized formats by developers that need to be taken into account – exercise for later. \n\n### Start with a simple log format \n\nInspect a running Linux VM, or use the following example log file example for additional implementation.\n\n```\nless /var/log/syslog | grep -v docker \n\nOct 17 00:00:04 ebpf-chaos systemd[1]: Starting Daily dpkg database backup service...\nOct 17 00:00:04 ebpf-chaos systemd[1]: Starting Rotate log files...\nOct 17 00:00:04 ebpf-chaos systemd[1]: dpkg-db-backup.service: Deactivated successfully.\nOct 17 00:00:04 ebpf-chaos systemd[1]: Finished Daily dpkg database backup service.\nOct 17 00:00:04 ebpf-chaos systemd[1]: logrotate.service: Deactivated successfully.\nOct 17 00:00:04 ebpf-chaos systemd[1]: Finished Rotate log files.\nOct 17 00:17:01 ebpf-chaos CRON[727495]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)\n```\n\nWe can create an algorithm to split each log line by whitespaces, and then join the results again. Let's ask Code Suggestions for help. \n\n```python\n# Split log line \"Oct 17 00:00:04 ebpf-chaos systemd[1]: Finished Rotate log files.\" by whitespaces and save in a list\n\nlog_line = \"Oct 17 00:00:04 ebpf-chaos systemd[1]: Finished Rotate log files.\"\nlog_line_split = log_line.split(\" \")\nprint(log_line_split)\n```\n\nRun the script again to verify the result.\n\n```shell\npython3 log_reader.py\n\n['Oct', '17', '00:00:04', 'ebpf-chaos', 'systemd[1]:', 'Finished', 'Rotate', 'log', 'files.']\n```\n\nThe first three items are part of the datetime string, followed by the host, service, and remaining log message items. Let's practice string operations in Python as the next step. \n\n### String and data structure operations\n\nLet's ask Code Suggestions for help with learning to join strings, and perform list operations.\n\n1. Join the first three items with a whitespace again. \n2. Keep host and service. \n3. Join the remaining variable item count into a string, separated with whitespaces, again. \n4. Store the identified column keys, and their respective values in a new data structure: [dictionary](https://docs.python.org/3/library/stdtypes.html#mapping-types-dict). \n\n![Code suggestions for list items with string operations](https://about.gitlab.com/images/blogimages/learn-python-with-ai-code-suggestions-getting-started/vs_code_code_suggestions_list_items_string_join_extract.png){: .shadow}\n\n```shell \npython3 log_reader.py\n\n# Array\n['Oct', '17', '00:00:04', 'ebpf-chaos', 'systemd[1]:', 'Finished', 'Rotate', 'log', 'files.']\n\n# Dictionary \n{'datetime': 'Oct 17 00:00:04', 'host': 'ebpf-chaos', 'service': 'systemd[1]:', 'message': ' ebpf-chaos systemd[1]: Finished Rotate log files.'}\n```\n\nA working suggestion can look like the following:\n\n```python\n# Initialize results dictionary with empty values for datetime, host, service, message\n# Loop over log line split \n# Join the first three list items as date string\n# Item 4: host \n# Item 5: service\n# Join the remaining items into a string, separated with whitespaces \n# Print the results after the loop \n\nresults = {'datetime': '', 'host': '', 'service': '', 'message': ''}\n\nfor item in log_line_split:\n\n    if results['datetime'] == '':\n        results['datetime'] = ' '.join(log_line_split[0:3])\n\n    elif results['host'] == '':\n        results['host'] = log_line_split[3]\n\n    elif results['service'] == '':\n        results['service'] = log_line_split[4]\n\n    else:\n        results['message'] += ' ' + item\n\nprint(results)\n\n```\n\nThe suggested algorithm loops over all log line items, and applies the same operation for the first three items. `log_line_split[0:3]` extracts a slice of three items into a new list. Calling `join()` on a separator character and passing the array as an argument joins the items into a string. The algorithm continues to check for not initialized values for host (Item 4) and service (Item 5)and concludes with the remaining list items appended into the message string. To be honest, I would have used a slightly different algorithm, but it is a great learning curve to see other algorithms, and ways to implement them. Practice with different instructions, and data structures, and continue printing the data sets. \n\n**Tip:** If you need to terminate a script early, you can use `sys.exit()`. The remaining code will not be executed. \n\n```python\nimport sys \nsys.exit(1)\n```\n\nImagine doing these operations for different log formats, and message types – it can get complicated and error-prone very quickly. Maybe there is another approach. \n\n### Parse log files using regular expressions\n\nThere are different syslog format RFCs – [RFC 3164](https://datatracker.ietf.org/doc/html/rfc3164) is obsolete but still found in the wild as default configuration (matching the pattern above), while [RFC 5424](https://datatracker.ietf.org/doc/html/rfc5424) is more modern, including datetime with timezone information. Parsing this format can be tricky, so let's ask Code Suggestions for advice. \n\nIn some cases, the suggestions include regular expressions. They might not match immediately, making the code more complex to debug, with trial and errors. A good standalone resource to text and explain regular expressions is [regex101.com](https://regex101.com/).  \n\n**Tip:** You can skip diving deep into regular expressions using the following code snippet as a quick cheat. The next step involves instructing Code Suggestions to use these log patterns, and help us extract all valuable columns. \n\n```python\n# Define the syslog log format regex in a dictionary\n# Add entries for RFC3164, RFC5424\nregex_log_pattern = {\n    'rfc3164': '([A-Z][a-z][a-z]\\s{1,2}\\d{1,2}\\s\\d{2}[:]\\d{2}[:]\\d{2})\\s([\\w][\\w\\d\\.@-]*)\\s(.*)$',\n    'rfc5424': '(?:(\\d{4}[-]\\d{2}[-]\\d{2}[T]\\d{2}[:]\\d{2}[:]\\d{2}(?:\\.\\d{1,6})?(?:[+-]\\d{2}[:]\\d{2}|Z)?)|-)\\s(?:([\\w][\\w\\d\\.@-]*)|-)\\s(.*)$;'\n}\n```\n\nWe know what the function should do, and its input parameters – the file name, and a log pattern to match. The log lines should be split by this regular expression, returning a key-value dictionary for each log line. The function should return a list of dictionaries. \n\n```python\n# Create a function that parses a log file\n# Input parameter: file path\n# Match log line against regex_log_pattern\n# Return the results as dictionary list: log line, pattern, extracted columns\n```\n\n![Code suggestion based on a multiline comment instruction to get a function that parses a log file based on regex patterns](https://about.gitlab.com/images/blogimages/learn-python-with-ai-code-suggestions-getting-started/vs_code_code_suggestions_log_format_regex_function_instructions_01.png){: .shadow}\n\nRemember the indent for opening a new scope? The same applies for functions in Python. The `def` identifier requires a function name, and a list of parameters, followed by an opening colon. The next lines of code require the indent. VS Code will help with live-linting wrong indent, before the script execution fails, or the CI/CD pipelines. \n\nContinue with Code Suggestions – it might already know that you want to parse all log files, and parse them using the newly created function. \n\n![Code suggestion to parse all log files, and print the result set](https://about.gitlab.com/images/blogimages/learn-python-with-ai-code-suggestions-getting-started/vs_code_code_suggestions_log_format_regex_function_instructions_02.png){: .shadow}\n\nA full working example can look like this: \n\n```\nimport os\n\n# Specify the path and file suffix in variables\npath = 'log-data/'\nfile_suffix = '.log'\n\n# Read all file paths in the directory into a list\nlog_files = []\nfor root, directories, files in os.walk(path):\n    for file in files:\n        if file.endswith(file_suffix):\n            log_files.append(os.path.join(root, file))\n\n# Define the syslog log format regex in a dictionary\n# Add entries for RFC3164, RFC5424\nregex_log_pattern = {\n    'rfc3164': '([A-Z][a-z][a-z]\\s{1,2}\\d{1,2}\\s\\d{2}[:]\\d{2}[:]\\d{2})\\s([\\w][\\w\\d\\.@-]*)\\s(.*)$',\n    'rfc5424': '(?:(\\d{4}[-]\\d{2}[-]\\d{2}[T]\\d{2}[:]\\d{2}[:]\\d{2}(?:\\.\\d{1,6})?(?:[+-]\\d{2}[:]\\d{2}|Z)?)|-)\\s(?:([\\w][\\w\\d\\.@-]*)|-)\\s(.*)$;'\n}\n\n# Create a function that parses a log file\n# Input parameter: file path\n# Match log line against regex_log_pattern\n# Return the results as dictionary list: log line, pattern name, extracted columns\nimport re\n\ndef parse_log_file(file_path):\n    # Read the log file\n    with open(file_path, 'r') as f:\n        log_lines = f.readlines()\n\n    # Create a list to store the results\n    results = []\n\n    # Iterate over the log lines\n    for log_line in log_lines:\n        # Match the log line against the regex pattern\n        for pattern_name, pattern in regex_log_pattern.items():\n            match = re.match(pattern, log_line)\n\n            # If the log line matches the pattern, add the results to the list\n            if match:\n                extracted_columns = match.groups()\n                results.append({\n                    'log_line': log_line,\n                    'pattern_name': pattern_name,\n                    'extracted_columns': extracted_columns,\n                    'source_file': file_path\n                })\n\n    # Return the results\n    return results\n\n# Parse all files and print results\nfor log_file in log_files:\n    results = parse_log_file(log_file)\n    print(results)\n```\n\nLet's unpack what the `parse_log_file()` function does:\n\n1. Opens the file from `file_path` parameter. \n2. Reads all lines into a new variable `log_lines`. \n3. Creates a results list to store all items. \n4. Iterates over the log lines. \n5. Matches against all regex patterns configured in regex_log_pattern. \n6. If a match is found, extracts the matching column values.\n7. Creates a results item, including the values for the keys `log_line`, `pattern_name`, `extracted_colums`, `source_file`. \n8. Appends the results item to the results list.\n9. Returns the results list. \n\nThere are different variations to this – especially for the returned result data structure. For this specific case, log lines come as list already. Adding a dictionary object instead of a raw log line allows function callers to extract the desired information in the next step. Once a working example has been implemented, you can refactor the code later, too. \n\n### Advanced log format: auth.log\n\nParsing the syslog on a Linux distribution might not unveil the necessary data to analyze. On a virtual machine that exposes port 22 (SSH) to the world, the authentication log is much more interesting – plenty of bots and malicious actors testing default password combinations and often brute force attacks.\n\nThe following snippet from `/var/log/auth.log` on one of my private servers shows the authentication log format and the random attempts from bots using different usernames, etc. \n\n```\nOct 15 00:00:19 ebpf-chaos sshd[3967944]: Failed password for invalid user ubuntu from 93.254.246.194 port 48840 ssh2\nOct 15 00:00:20 ebpf-chaos sshd[3967916]: Failed password for root from 180.101.88.227 port 44397 ssh2\nOct 15 00:00:21 ebpf-chaos sshd[3967944]: Received disconnect from 93.254.246.194 port 48840:11: Bye Bye [preauth]\nOct 15 00:00:21 ebpf-chaos sshd[3967944]: Disconnected from invalid user ubuntu 93.254.246.194 port 48840 [preauth]\nOct 15 00:00:24 ebpf-chaos sshd[3967916]: Failed password for root from 180.101.88.227 port 44397 ssh2\nOct 15 00:00:25 ebpf-chaos sshd[3967916]: Received disconnect from 180.101.88.227 port 44397:11:  [preauth]\nOct 15 00:00:25 ebpf-chaos sshd[3967916]: Disconnected from authenticating user root 180.101.88.227 port 44397 [preauth]\nOct 15 00:00:25 ebpf-chaos sshd[3967916]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=180.101.88.227  user=root\nOct 15 00:00:25 ebpf-chaos sshd[3967998]: Invalid user teamspeak from 185.218.20.10 port 33436\n```\n\n**Tip for intrusion prevention:** Add a firewall setup, and use [fail2ban](https://en.wikipedia.org/wiki/Fail2ban) to block invalid auth logins. \n\nThe next exercise is to extend the logic to understand the free form log message parts, for example `Failed password for invalid user ubuntu from 93.254.246.194 port 48840 ssh2`. The task is to store the data in an optional dictionary with key value pairs. \n\nCreate a new function that takes the previously parsed log line results as input, and specifically parses the last list item for each line.\n\n1. Count the number of `Failed password` and `Invalid user` messages.\n2. Return the results with count, log file, pattern \n\n![Code suggestions for a log file message parser to count auth.log failures](https://about.gitlab.com/images/blogimages/learn-python-with-ai-code-suggestions-getting-started/vs_code_code_suggestions_parse_log_message_auth_log.png){: .shadow}\n\nA working suggestion can look like the following code:\n\n```python\n# Create a function that parses a log file message from the last extracted_columns entry \n# Input: Parsed log lines results list \n# Loop over all log lines in the list, and extract the last list item as message \n# Count failure strings in the message: Failed password, Invalid user \n# Return the results if failure count greater 0: log_file, count, failure string\ndef parse_log_file_message(results):\n    failure_results = []\n\n    # Iterate over the log lines\n    for result in results:\n        # Extract the message from the last list item\n        message = result['extracted_columns'][-1]\n\n        # Count the number of failure strings in the message\n        failure_count = message.count('Failed password') + message.count('Invalid user')\n\n        # If the failure count is greater than 0, add the results to the list\n        if failure_count > 0:\n            failure_results.append({\n                'log_file': result['source_file'],\n                'count': failure_count,\n                'failure_string': message\n            })\n\n    # Return the results\n    return failure_results\n\n# Parse all files and print results\nfor log_file in log_files:\n    results = parse_log_file(log_file)\n    failure_results = parse_log_file_message(results)\n    print(failure_results)\n```\n\nThe algorithm follows the previous implementations: First, create a results array to store matching data. Then, iterate over the already parsed log_lines in the list. Each log line contains the `extracted_columns` key, which holds the free-form message string at the end. The next step is to call the string object function `count()` to count how many times a given character sequence is contained in a string. The returned numbers are added up to the `failure_count` variable. If it is greater than zero, the result is added to the results list, including the `log_file`, `count` and `failure_string` key-value pairs. After returning the parsed log message results, loop through all log files, parse them, and print the results again. \n\nExecute the script to inspect the detected matches. Note that the data structure can be optimized in future learning steps.\n\n```\npython3 log_reader.py\n\n[{'log_file': 'log-data/var/log/auth.log', 'count': 1, 'failure_string': 'sshd[3967944]: Failed password for invalid user ubuntu from 93.254.246.194 port 48840 ssh2'}, {'log_file': 'log-data/var/log/auth.log', 'count': 1, 'failure_string': 'sshd[3967916]: Failed password for root from 180.101.88.227 port 44397 ssh2'}, {'log_file': 'log-data/var/log/auth.log', 'count': 1, 'failure_string': 'sshd[3967916]: Failed password for root from 180.101.88.227 port 44397 ssh2'}, {'log_file': 'log-data/var/log/auth.log', 'count': 1, 'failure_string': 'sshd[3967998]: Invalid user teamspeak from 185.218.20.10 port 33436'}, {'log_file': 'log-data/var/log/auth.log', 'count': 1, 'failure_string': 'sshd[3967998]: Failed password for invalid user teamspeak from 185.218.20.10 port 33436 ssh2'}, {'log_file': 'log-data/var/log/auth.log', 'count': 1, 'failure_string': 'sshd[3968077]: Invalid user mcserver from 218.211.33.146 port 50950'}]\n\n```\n\n### Parsing more types: Structured logging\n\nApplication developers can use the structured logging format to help machine parsers to extract the key value pairs. Prometheus provides this information in the following structure in syslog:\n\n```\nOct 17 19:00:10 ebpf-chaos prometheus[594]: ts=2023-10-17T19:00:10.425Z caller=compact.go:519 level=info component=tsdb m\nsg=\"write block\" mint=1697558404661 maxt=1697565600000 ulid=01HCZG4ZX51GTH8H7PVBYDF4N6 duration=148.675854ms\nOct 17 19:00:10 ebpf-chaos prometheus[594]: ts=2023-10-17T19:00:10.464Z caller=head.go:1213 level=info component=tsdb msg\n=\"Head GC completed\" caller=truncateMemory duration=6.845245ms\nOct 17 19:00:10 ebpf-chaos prometheus[594]: ts=2023-10-17T19:00:10.467Z caller=checkpoint.go:100 level=info component=tsd\nb msg=\"Creating checkpoint\" from_segment=2308 to_segment=2309 mint=1697565600000\nOct 17 19:00:10 ebpf-chaos prometheus[594]: ts=2023-10-17T19:00:10.517Z caller=head.go:1185 level=info component=tsdb msg\n=\"WAL checkpoint complete\" first=2308 last=2309 duration=50.052621ms\n```\n\nThis format is easier to parse for scripts, because the message part can be split by whitespaces, and the assignment character `=`. Strings that contain whitespaces are guaranteed to be enclosed with quotes. The downside is that not all programming language libraries provide ready-to-use structured logging libraries, making it harder for developers to adopt this format. \n\nPractice following the previous example to parse the `auth.log` format with additional information. Tell Code Suggestions that you are expecting structured logging format with key-value pairs, and which returned data structure would be great:\n\n```python\n# Create a function that parses a log file message from the last extracted_columns entry \n# Input: Parsed log lines results list \n# Loop over all log lines in the list, and extract the last list item as message \n# Parse structured logging key-value pairs into a dictionary\n# Return results: log_file, dictionary \n```\n\n![Code suggestions for parsing structured logging format in the log file message part](https://about.gitlab.com/images/blogimages/learn-python-with-ai-code-suggestions-getting-started/vs_code_code_suggestions_propose_structured_logging_message_parser.png){: .shadow}\n\n### Printing results and formatting\n\nMany of the examples used the `print()` statement to print the content on the terminal. Python objects in the standard library support text representation, and for some types it makes more sense (string, numbers), others cannot provide much details (functions, etc.). \n\nYou can also pretty-print almost any data structure (lists, sets, dictionaries) in Python. The JSON library can format data structures in a readable format, and use a given spaces indent to draw the JSON structure on the terminal. Note that we use the `import` statement here to bring libraries into the current scope, and access their methods, for example `json.dumps`. \n\n```python\nimport json \nprint(json.dumps(structured_results, indent=4))\n```\n\n![Parsing log files into structured objects, example result after following the exercises](https://about.gitlab.com/images/blogimages/learn-python-with-ai-code-suggestions-getting-started/vs_code_terminal_parsing_logs_and_pretty_print_results.png)\n\nPractice with modifying the existing source code, and replace the code snippets where appropriate. Alternatively, create a new function that implements pretty printing.\n\n```python\n# Create a pretty print function with indent 4 \n```\n\n![Code suggestions for pretty-print function](https://about.gitlab.com/images/blogimages/learn-python-with-ai-code-suggestions-getting-started/vs_code_code_suggestions_pretty_print.png){: .shadow}\n\nThis idea works in a similar fashion with creating your own logger functions...but we have to stop learning and take a break. Before we conclude the first blog post in the learning series, let's ensure that CI/CD and dependencies are set up properly for future exercises and async practice. \n\n## Dependency management and continuous verification  \n\n### Pip and pyenv: Bringing structure into Python \n\nDependencies can be managed in the [`requirements.txt` file](https://pip.pypa.io/en/stable/reference/requirements-file-format/), including optional version dependencies. Using `requirements.txt` file also has the advantage of being the single source of truth for local development environments and running continuous builds with GitLab CI/CD. They can use the same installation command:\n\n```shell\npip install -r requirements.txt\n```\n\nSome Linux distributions do not install the pip package manager by default, for example, Ubuntu/Debian require to install the `python3-pip` package. \n\nYou can manage different virtual environments using [venv](https://docs.python.org/3/library/venv.html). This workflow can be beneficial to install Python dependencies into the virtual environment, instead of globally into the OS path which might break on upgrades. \n\n```shell\npip install virtualenv\nvirtualenv venv\nsource venv/bin/activate \n```\n\n### Automation: Configure CI/CD pipeline for Python\n\nThe [CI/CD pipeline](https://docs.gitlab.com/ee/ci/) should continuously lint, test, and build the code. You can mimic the steps from the local development, and add testing more environments and versions: \n\n1. Lint the source code and check for formatting errors. The example uses [Pyflakes](https://pypi.org/project/pyflakes/), a mature linter, and [Ruff](https://docs.astral.sh/ruff/ ), a fast linter written in Rust. \n2. Cache dependencies installed using the pip package manager, following the documentation for [Python caching in GitLab CI/CD](https://docs.gitlab.com/ee/ci/caching/#cache-python-dependencies). This saves time and resources on repeated CI/CD pipeline runs.\n3. Use parallel matrix builds to test different Python versions, based on the available container images on Docker Hub and their tags. \n\n```yaml\nstages:\n  - lint\n  - test\n\ndefault:\n  image: python:latest\n  cache:                      # Pip's cache doesn't store the python packages\n    paths:                    # https://pip.pypa.io/en/stable/topics/caching/\n      - .cache/pip\n  before_script:\n    - python -V               # Print out python version for debugging\n    - pip install virtualenv\n    - virtualenv venv\n    - source venv/bin/activate\n\nvariables:  # Change pip's cache directory to be inside the project directory since we can only cache local items.\n  PIP_CACHE_DIR: \"$CI_PROJECT_DIR/.cache/pip\"\n\n# lint template\n.lint-tmpl:\n  script:\n    - echo \"Linting Python version $VERSION\"\n  parallel:\n    matrix:\n      - VERSION: ['3.9', '3.10', '3.11', '3.12']   # https://hub.docker.com/_/python\n\n# Lint, using Pyflakes: https://pypi.org/project/pyflakes/ \nlint-pyflakes:\n  extends: [.lint-tmpl]\n  script:\n    - pip install -r requirements.txt\n    - find . -not -path './venv' -type f -name '*.py' -exec sh -c 'pyflakes {}' \\;\n\n# Lint, using Ruff (Rust): https://docs.astral.sh/ruff/ \nlint-ruff:\n  extends: [.lint-tmpl]\n  script:\n    - pip install -r requirements.txt\n    - ruff .\n```\n\n![GitLab CI/CD Python lint job view, part of matrix builds](https://about.gitlab.com/images/blogimages/learn-python-with-ai-code-suggestions-getting-started/gitlab_cicd_python_lint_job_log_view.png)\n\n## What is next \n\nFun fact: GitLab Duo Code Suggestions also helped writing this blog post in VS Code, knowing about the context. In the screenshot, I just wanted to add a tip about [regex101](https://regex101.com/), and GitLab Duo already knew. \n\n![Writing the GitLab blog post in VS Code with support from GitLab Duo Code Suggestions](https://about.gitlab.com/images/blogimages/learn-python-with-ai-code-suggestions-getting-started/gitlab_duo_code_suggestions_helping_write_the_learning_python_ai_blog_post.png)\n\nIn an upcoming blog, we will look into advanced learning examples with more practical (log) filtering and parallel operations, how to fetch logs from API endpoints (CI/CD job logs for example), and more data analytics and observability. Until then, here are a few recommendations for practicing async.\n\n### Async learning exercises\n\n- Implement the missing `log_file_limit` variable check. \n- Print a summary of the results in Markdown, not only JSON format. \n- Extend the script to accept a search filter as environment variable. Print/count only filtered results. \n- Extend the script to accept a date range. It might require parsing the datetime column in a time object to compare the range. \n- Inspect a GitLab CI/CD pipeline job log, and download the raw format. Extend the log parser to parse this specific format, and print a summary. \n\n### Share your feedback\n\nWhich programming language are you learning or considering learning? Start a new topic on our [community](/community/) forum or Discord and share your experience.\n\nWhen you use [GitLab Duo](/gitlab-duo/) Code Suggestions, please share your thoughts and feedback [in the feedback issue](https://gitlab.com/gitlab-org/gitlab/-/issues/405152).\n","ai-ml",[23,24,25,26],"DevSecOps platform","tutorial","workflow","AI/ML",{"slug":28,"featured":6,"template":29},"learning-python-with-a-little-help-from-ai-code-suggestions","BlogPost","content:en-us:blog:learning-python-with-a-little-help-from-ai-code-suggestions.yml","yaml","Learning Python With A Little Help From Ai Code Suggestions","content","en-us/blog/learning-python-with-a-little-help-from-ai-code-suggestions.yml","en-us/blog/learning-python-with-a-little-help-from-ai-code-suggestions","yml",{"_path":38,"_dir":39,"_draft":6,"_partial":6,"_locale":7,"data":40,"_id":450,"_type":31,"title":451,"_source":33,"_file":452,"_stem":453,"_extension":36},"/shared/en-us/main-navigation","en-us",{"logo":41,"freeTrial":46,"sales":51,"login":56,"items":61,"search":391,"minimal":422,"duo":441},{"config":42},{"href":43,"dataGaName":44,"dataGaLocation":45},"/","gitlab logo","header",{"text":47,"config":48},"Get free trial",{"href":49,"dataGaName":50,"dataGaLocation":45},"https://gitlab.com/-/trial_registrations/new?glm_source=about.gitlab.com&glm_content=default-saas-trial/","free trial",{"text":52,"config":53},"Talk to sales",{"href":54,"dataGaName":55,"dataGaLocation":45},"/sales/","sales",{"text":57,"config":58},"Sign in",{"href":59,"dataGaName":60,"dataGaLocation":45},"https://gitlab.com/users/sign_in/","sign in",[62,106,202,207,312,372],{"text":63,"config":64,"cards":66,"footer":89},"Platform",{"dataNavLevelOne":65},"platform",[67,73,81],{"title":63,"description":68,"link":69},"The most comprehensive AI-powered DevSecOps Platform",{"text":70,"config":71},"Explore our Platform",{"href":72,"dataGaName":65,"dataGaLocation":45},"/platform/",{"title":74,"description":75,"link":76},"GitLab Duo (AI)","Build software faster with AI at every stage of development",{"text":77,"config":78},"Meet GitLab Duo",{"href":79,"dataGaName":80,"dataGaLocation":45},"/gitlab-duo/","gitlab duo ai",{"title":82,"description":83,"link":84},"Why GitLab","10 reasons why Enterprises choose GitLab",{"text":85,"config":86},"Learn more",{"href":87,"dataGaName":88,"dataGaLocation":45},"/why-gitlab/","why gitlab",{"title":90,"items":91},"Get started with",[92,97,102],{"text":93,"config":94},"Platform Engineering",{"href":95,"dataGaName":96,"dataGaLocation":45},"/solutions/platform-engineering/","platform engineering",{"text":98,"config":99},"Developer Experience",{"href":100,"dataGaName":101,"dataGaLocation":45},"/developer-experience/","Developer experience",{"text":103,"config":104},"MLOps",{"href":105,"dataGaName":103,"dataGaLocation":45},"/topics/devops/the-role-of-ai-in-devops/",{"text":107,"left":108,"config":109,"link":111,"lists":115,"footer":184},"Product",true,{"dataNavLevelOne":110},"solutions",{"text":112,"config":113},"View all Solutions",{"href":114,"dataGaName":110,"dataGaLocation":45},"/solutions/",[116,141,163],{"title":117,"description":118,"link":119,"items":124},"Automation","CI/CD and automation to accelerate deployment",{"config":120},{"icon":121,"href":122,"dataGaName":123,"dataGaLocation":45},"AutomatedCodeAlt","/solutions/delivery-automation/","automated software delivery",[125,129,133,137],{"text":126,"config":127},"CI/CD",{"href":128,"dataGaLocation":45,"dataGaName":126},"/solutions/continuous-integration/",{"text":130,"config":131},"AI-Assisted Development",{"href":79,"dataGaLocation":45,"dataGaName":132},"AI assisted development",{"text":134,"config":135},"Source Code Management",{"href":136,"dataGaLocation":45,"dataGaName":134},"/solutions/source-code-management/",{"text":138,"config":139},"Automated Software Delivery",{"href":122,"dataGaLocation":45,"dataGaName":140},"Automated software delivery",{"title":142,"description":143,"link":144,"items":149},"Security","Deliver code faster without compromising security",{"config":145},{"href":146,"dataGaName":147,"dataGaLocation":45,"icon":148},"/solutions/security-compliance/","security and compliance","ShieldCheckLight",[150,153,158],{"text":151,"config":152},"Security & Compliance",{"href":146,"dataGaLocation":45,"dataGaName":151},{"text":154,"config":155},"Software Supply Chain Security",{"href":156,"dataGaLocation":45,"dataGaName":157},"/solutions/supply-chain/","Software supply chain security",{"text":159,"config":160},"Compliance & Governance",{"href":161,"dataGaLocation":45,"dataGaName":162},"/solutions/continuous-software-compliance/","Compliance and governance",{"title":164,"link":165,"items":170},"Measurement",{"config":166},{"icon":167,"href":168,"dataGaName":169,"dataGaLocation":45},"DigitalTransformation","/solutions/visibility-measurement/","visibility and measurement",[171,175,179],{"text":172,"config":173},"Visibility & Measurement",{"href":168,"dataGaLocation":45,"dataGaName":174},"Visibility and Measurement",{"text":176,"config":177},"Value Stream Management",{"href":178,"dataGaLocation":45,"dataGaName":176},"/solutions/value-stream-management/",{"text":180,"config":181},"Analytics & Insights",{"href":182,"dataGaLocation":45,"dataGaName":183},"/solutions/analytics-and-insights/","Analytics and insights",{"title":185,"items":186},"GitLab for",[187,192,197],{"text":188,"config":189},"Enterprise",{"href":190,"dataGaLocation":45,"dataGaName":191},"/enterprise/","enterprise",{"text":193,"config":194},"Small Business",{"href":195,"dataGaLocation":45,"dataGaName":196},"/small-business/","small business",{"text":198,"config":199},"Public Sector",{"href":200,"dataGaLocation":45,"dataGaName":201},"/solutions/public-sector/","public sector",{"text":203,"config":204},"Pricing",{"href":205,"dataGaName":206,"dataGaLocation":45,"dataNavLevelOne":206},"/pricing/","pricing",{"text":208,"config":209,"link":211,"lists":215,"feature":299},"Resources",{"dataNavLevelOne":210},"resources",{"text":212,"config":213},"View all resources",{"href":214,"dataGaName":210,"dataGaLocation":45},"/resources/",[216,249,271],{"title":217,"items":218},"Getting started",[219,224,229,234,239,244],{"text":220,"config":221},"Install",{"href":222,"dataGaName":223,"dataGaLocation":45},"/install/","install",{"text":225,"config":226},"Quick start guides",{"href":227,"dataGaName":228,"dataGaLocation":45},"/get-started/","quick setup checklists",{"text":230,"config":231},"Learn",{"href":232,"dataGaLocation":45,"dataGaName":233},"https://university.gitlab.com/","learn",{"text":235,"config":236},"Product documentation",{"href":237,"dataGaName":238,"dataGaLocation":45},"https://docs.gitlab.com/","product documentation",{"text":240,"config":241},"Best practice videos",{"href":242,"dataGaName":243,"dataGaLocation":45},"/getting-started-videos/","best practice videos",{"text":245,"config":246},"Integrations",{"href":247,"dataGaName":248,"dataGaLocation":45},"/integrations/","integrations",{"title":250,"items":251},"Discover",[252,257,261,266],{"text":253,"config":254},"Customer success stories",{"href":255,"dataGaName":256,"dataGaLocation":45},"/customers/","customer success stories",{"text":258,"config":259},"Blog",{"href":260,"dataGaName":5,"dataGaLocation":45},"/blog/",{"text":262,"config":263},"Remote",{"href":264,"dataGaName":265,"dataGaLocation":45},"https://handbook.gitlab.com/handbook/company/culture/all-remote/","remote",{"text":267,"config":268},"TeamOps",{"href":269,"dataGaName":270,"dataGaLocation":45},"/teamops/","teamops",{"title":272,"items":273},"Connect",[274,279,284,289,294],{"text":275,"config":276},"GitLab Services",{"href":277,"dataGaName":278,"dataGaLocation":45},"/services/","services",{"text":280,"config":281},"Community",{"href":282,"dataGaName":283,"dataGaLocation":45},"/community/","community",{"text":285,"config":286},"Forum",{"href":287,"dataGaName":288,"dataGaLocation":45},"https://forum.gitlab.com/","forum",{"text":290,"config":291},"Events",{"href":292,"dataGaName":293,"dataGaLocation":45},"/events/","events",{"text":295,"config":296},"Partners",{"href":297,"dataGaName":298,"dataGaLocation":45},"/partners/","partners",{"backgroundColor":300,"textColor":301,"text":302,"image":303,"link":307},"#2f2a6b","#fff","Insights for the future of software development",{"altText":304,"config":305},"the source promo card",{"src":306},"/images/navigation/the-source-promo-card.svg",{"text":308,"config":309},"Read the latest",{"href":310,"dataGaName":311,"dataGaLocation":45},"/the-source/","the source",{"text":313,"config":314,"lists":316},"Company",{"dataNavLevelOne":315},"company",[317],{"items":318},[319,324,330,332,337,342,347,352,357,362,367],{"text":320,"config":321},"About",{"href":322,"dataGaName":323,"dataGaLocation":45},"/company/","about",{"text":325,"config":326,"footerGa":329},"Jobs",{"href":327,"dataGaName":328,"dataGaLocation":45},"/jobs/","jobs",{"dataGaName":328},{"text":290,"config":331},{"href":292,"dataGaName":293,"dataGaLocation":45},{"text":333,"config":334},"Leadership",{"href":335,"dataGaName":336,"dataGaLocation":45},"/company/team/e-group/","leadership",{"text":338,"config":339},"Team",{"href":340,"dataGaName":341,"dataGaLocation":45},"/company/team/","team",{"text":343,"config":344},"Handbook",{"href":345,"dataGaName":346,"dataGaLocation":45},"https://handbook.gitlab.com/","handbook",{"text":348,"config":349},"Investor relations",{"href":350,"dataGaName":351,"dataGaLocation":45},"https://ir.gitlab.com/","investor relations",{"text":353,"config":354},"Trust Center",{"href":355,"dataGaName":356,"dataGaLocation":45},"/security/","trust center",{"text":358,"config":359},"AI Transparency Center",{"href":360,"dataGaName":361,"dataGaLocation":45},"/ai-transparency-center/","ai transparency center",{"text":363,"config":364},"Newsletter",{"href":365,"dataGaName":366,"dataGaLocation":45},"/company/contact/","newsletter",{"text":368,"config":369},"Press",{"href":370,"dataGaName":371,"dataGaLocation":45},"/press/","press",{"text":373,"config":374,"lists":375},"Contact us",{"dataNavLevelOne":315},[376],{"items":377},[378,381,386],{"text":52,"config":379},{"href":54,"dataGaName":380,"dataGaLocation":45},"talk to sales",{"text":382,"config":383},"Get help",{"href":384,"dataGaName":385,"dataGaLocation":45},"/support/","get help",{"text":387,"config":388},"Customer portal",{"href":389,"dataGaName":390,"dataGaLocation":45},"https://customers.gitlab.com/customers/sign_in/","customer portal",{"close":392,"login":393,"suggestions":400},"Close",{"text":394,"link":395},"To search repositories and projects, login to",{"text":396,"config":397},"gitlab.com",{"href":59,"dataGaName":398,"dataGaLocation":399},"search login","search",{"text":401,"default":402},"Suggestions",[403,405,409,411,415,419],{"text":74,"config":404},{"href":79,"dataGaName":74,"dataGaLocation":399},{"text":406,"config":407},"Code Suggestions (AI)",{"href":408,"dataGaName":406,"dataGaLocation":399},"/solutions/code-suggestions/",{"text":126,"config":410},{"href":128,"dataGaName":126,"dataGaLocation":399},{"text":412,"config":413},"GitLab on AWS",{"href":414,"dataGaName":412,"dataGaLocation":399},"/partners/technology-partners/aws/",{"text":416,"config":417},"GitLab on Google Cloud",{"href":418,"dataGaName":416,"dataGaLocation":399},"/partners/technology-partners/google-cloud-platform/",{"text":420,"config":421},"Why GitLab?",{"href":87,"dataGaName":420,"dataGaLocation":399},{"freeTrial":423,"mobileIcon":428,"desktopIcon":433,"secondaryButton":436},{"text":424,"config":425},"Start free trial",{"href":426,"dataGaName":50,"dataGaLocation":427},"https://gitlab.com/-/trials/new/","nav",{"altText":429,"config":430},"Gitlab Icon",{"src":431,"dataGaName":432,"dataGaLocation":427},"/images/brand/gitlab-logo-tanuki.svg","gitlab icon",{"altText":429,"config":434},{"src":435,"dataGaName":432,"dataGaLocation":427},"/images/brand/gitlab-logo-type.svg",{"text":437,"config":438},"Get Started",{"href":439,"dataGaName":440,"dataGaLocation":427},"https://gitlab.com/-/trial_registrations/new?glm_source=about.gitlab.com/compare/gitlab-vs-github/","get started",{"freeTrial":442,"mobileIcon":446,"desktopIcon":448},{"text":443,"config":444},"Learn more about GitLab Duo",{"href":79,"dataGaName":445,"dataGaLocation":427},"gitlab duo",{"altText":429,"config":447},{"src":431,"dataGaName":432,"dataGaLocation":427},{"altText":429,"config":449},{"src":435,"dataGaName":432,"dataGaLocation":427},"content:shared:en-us:main-navigation.yml","Main Navigation","shared/en-us/main-navigation.yml","shared/en-us/main-navigation",{"_path":455,"_dir":39,"_draft":6,"_partial":6,"_locale":7,"title":456,"button":457,"config":461,"_id":463,"_type":31,"_source":33,"_file":464,"_stem":465,"_extension":36},"/shared/en-us/banner","GitLab Duo Agent Platform is now in public beta!",{"text":85,"config":458},{"href":459,"dataGaName":460,"dataGaLocation":45},"/gitlab-duo/agent-platform/","duo banner",{"layout":462},"release","content:shared:en-us:banner.yml","shared/en-us/banner.yml","shared/en-us/banner",{"_path":467,"_dir":39,"_draft":6,"_partial":6,"_locale":7,"data":468,"_id":672,"_type":31,"title":673,"_source":33,"_file":674,"_stem":675,"_extension":36},"/shared/en-us/main-footer",{"text":469,"source":470,"edit":476,"contribute":481,"config":486,"items":491,"minimal":664},"Git is a trademark of Software Freedom Conservancy and our use of 'GitLab' is under license",{"text":471,"config":472},"View page source",{"href":473,"dataGaName":474,"dataGaLocation":475},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/","page source","footer",{"text":477,"config":478},"Edit this page",{"href":479,"dataGaName":480,"dataGaLocation":475},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/-/blob/main/content/","web ide",{"text":482,"config":483},"Please contribute",{"href":484,"dataGaName":485,"dataGaLocation":475},"https://gitlab.com/gitlab-com/marketing/digital-experience/about-gitlab-com/-/blob/main/CONTRIBUTING.md/","please contribute",{"twitter":487,"facebook":488,"youtube":489,"linkedin":490},"https://twitter.com/gitlab","https://www.facebook.com/gitlab","https://www.youtube.com/channel/UCnMGQ8QHMAnVIsI3xJrihhg","https://www.linkedin.com/company/gitlab-com",[492,514,571,600,634],{"title":63,"links":493,"subMenu":497},[494],{"text":23,"config":495},{"href":72,"dataGaName":496,"dataGaLocation":475},"devsecops platform",[498],{"title":203,"links":499},[500,504,509],{"text":501,"config":502},"View plans",{"href":205,"dataGaName":503,"dataGaLocation":475},"view plans",{"text":505,"config":506},"Why Premium?",{"href":507,"dataGaName":508,"dataGaLocation":475},"/pricing/premium/","why premium",{"text":510,"config":511},"Why Ultimate?",{"href":512,"dataGaName":513,"dataGaLocation":475},"/pricing/ultimate/","why ultimate",{"title":515,"links":516},"Solutions",[517,522,525,527,532,537,541,544,548,553,555,558,561,566],{"text":518,"config":519},"Digital transformation",{"href":520,"dataGaName":521,"dataGaLocation":475},"/topics/digital-transformation/","digital transformation",{"text":151,"config":523},{"href":146,"dataGaName":524,"dataGaLocation":475},"security & compliance",{"text":140,"config":526},{"href":122,"dataGaName":123,"dataGaLocation":475},{"text":528,"config":529},"Agile development",{"href":530,"dataGaName":531,"dataGaLocation":475},"/solutions/agile-delivery/","agile delivery",{"text":533,"config":534},"Cloud transformation",{"href":535,"dataGaName":536,"dataGaLocation":475},"/topics/cloud-native/","cloud transformation",{"text":538,"config":539},"SCM",{"href":136,"dataGaName":540,"dataGaLocation":475},"source code management",{"text":126,"config":542},{"href":128,"dataGaName":543,"dataGaLocation":475},"continuous integration & delivery",{"text":545,"config":546},"Value stream management",{"href":178,"dataGaName":547,"dataGaLocation":475},"value stream management",{"text":549,"config":550},"GitOps",{"href":551,"dataGaName":552,"dataGaLocation":475},"/solutions/gitops/","gitops",{"text":188,"config":554},{"href":190,"dataGaName":191,"dataGaLocation":475},{"text":556,"config":557},"Small business",{"href":195,"dataGaName":196,"dataGaLocation":475},{"text":559,"config":560},"Public sector",{"href":200,"dataGaName":201,"dataGaLocation":475},{"text":562,"config":563},"Education",{"href":564,"dataGaName":565,"dataGaLocation":475},"/solutions/education/","education",{"text":567,"config":568},"Financial services",{"href":569,"dataGaName":570,"dataGaLocation":475},"/solutions/finance/","financial services",{"title":208,"links":572},[573,575,577,579,582,584,586,588,590,592,594,596,598],{"text":220,"config":574},{"href":222,"dataGaName":223,"dataGaLocation":475},{"text":225,"config":576},{"href":227,"dataGaName":228,"dataGaLocation":475},{"text":230,"config":578},{"href":232,"dataGaName":233,"dataGaLocation":475},{"text":235,"config":580},{"href":237,"dataGaName":581,"dataGaLocation":475},"docs",{"text":258,"config":583},{"href":260,"dataGaName":5,"dataGaLocation":475},{"text":253,"config":585},{"href":255,"dataGaName":256,"dataGaLocation":475},{"text":262,"config":587},{"href":264,"dataGaName":265,"dataGaLocation":475},{"text":275,"config":589},{"href":277,"dataGaName":278,"dataGaLocation":475},{"text":267,"config":591},{"href":269,"dataGaName":270,"dataGaLocation":475},{"text":280,"config":593},{"href":282,"dataGaName":283,"dataGaLocation":475},{"text":285,"config":595},{"href":287,"dataGaName":288,"dataGaLocation":475},{"text":290,"config":597},{"href":292,"dataGaName":293,"dataGaLocation":475},{"text":295,"config":599},{"href":297,"dataGaName":298,"dataGaLocation":475},{"title":313,"links":601},[602,604,606,608,610,612,614,618,623,625,627,629],{"text":320,"config":603},{"href":322,"dataGaName":315,"dataGaLocation":475},{"text":325,"config":605},{"href":327,"dataGaName":328,"dataGaLocation":475},{"text":333,"config":607},{"href":335,"dataGaName":336,"dataGaLocation":475},{"text":338,"config":609},{"href":340,"dataGaName":341,"dataGaLocation":475},{"text":343,"config":611},{"href":345,"dataGaName":346,"dataGaLocation":475},{"text":348,"config":613},{"href":350,"dataGaName":351,"dataGaLocation":475},{"text":615,"config":616},"Sustainability",{"href":617,"dataGaName":615,"dataGaLocation":475},"/sustainability/",{"text":619,"config":620},"Diversity, inclusion and belonging (DIB)",{"href":621,"dataGaName":622,"dataGaLocation":475},"/diversity-inclusion-belonging/","Diversity, inclusion and belonging",{"text":353,"config":624},{"href":355,"dataGaName":356,"dataGaLocation":475},{"text":363,"config":626},{"href":365,"dataGaName":366,"dataGaLocation":475},{"text":368,"config":628},{"href":370,"dataGaName":371,"dataGaLocation":475},{"text":630,"config":631},"Modern Slavery Transparency Statement",{"href":632,"dataGaName":633,"dataGaLocation":475},"https://handbook.gitlab.com/handbook/legal/modern-slavery-act-transparency-statement/","modern slavery transparency statement",{"title":635,"links":636},"Contact Us",[637,640,642,644,649,654,659],{"text":638,"config":639},"Contact an expert",{"href":54,"dataGaName":55,"dataGaLocation":475},{"text":382,"config":641},{"href":384,"dataGaName":385,"dataGaLocation":475},{"text":387,"config":643},{"href":389,"dataGaName":390,"dataGaLocation":475},{"text":645,"config":646},"Status",{"href":647,"dataGaName":648,"dataGaLocation":475},"https://status.gitlab.com/","status",{"text":650,"config":651},"Terms of use",{"href":652,"dataGaName":653,"dataGaLocation":475},"/terms/","terms of use",{"text":655,"config":656},"Privacy statement",{"href":657,"dataGaName":658,"dataGaLocation":475},"/privacy/","privacy statement",{"text":660,"config":661},"Cookie preferences",{"dataGaName":662,"dataGaLocation":475,"id":663,"isOneTrustButton":108},"cookie preferences","ot-sdk-btn",{"items":665},[666,668,670],{"text":650,"config":667},{"href":652,"dataGaName":653,"dataGaLocation":475},{"text":655,"config":669},{"href":657,"dataGaName":658,"dataGaLocation":475},{"text":660,"config":671},{"dataGaName":662,"dataGaLocation":475,"id":663,"isOneTrustButton":108},"content:shared:en-us:main-footer.yml","Main Footer","shared/en-us/main-footer.yml","shared/en-us/main-footer",[677],{"_path":678,"_dir":679,"_draft":6,"_partial":6,"_locale":7,"content":680,"config":684,"_id":686,"_type":31,"title":18,"_source":33,"_file":687,"_stem":688,"_extension":36},"/en-us/blog/authors/michael-friedrich","authors",{"name":18,"config":681},{"headshot":682,"ctfId":683},"https://res.cloudinary.com/about-gitlab-com/image/upload/v1749659879/Blog/Author%20Headshots/dnsmichi-headshot.jpg","dnsmichi",{"template":685},"BlogAuthor","content:en-us:blog:authors:michael-friedrich.yml","en-us/blog/authors/michael-friedrich.yml","en-us/blog/authors/michael-friedrich",{"_path":690,"_dir":39,"_draft":6,"_partial":6,"_locale":7,"header":691,"eyebrow":692,"blurb":693,"button":694,"secondaryButton":698,"_id":700,"_type":31,"title":701,"_source":33,"_file":702,"_stem":703,"_extension":36},"/shared/en-us/next-steps","Start shipping better software faster","50%+ of the Fortune 100 trust GitLab","See what your team can do with the intelligent\n\n\nDevSecOps platform.\n",{"text":47,"config":695},{"href":696,"dataGaName":50,"dataGaLocation":697},"https://gitlab.com/-/trial_registrations/new?glm_content=default-saas-trial&glm_source=about.gitlab.com/","feature",{"text":52,"config":699},{"href":54,"dataGaName":55,"dataGaLocation":697},"content:shared:en-us:next-steps.yml","Next Steps","shared/en-us/next-steps.yml","shared/en-us/next-steps",1753475333230]