How to Build a CLI Version of MonkeyType

CLI Version of MonkeyType

Type annotations have become one of the key tools in Python for boosting code quality, readability and enabling static analysis. However, this puts a burden on you as annotating these manually can be quite exhaustive especially in big codebases. To this end, the idea behind tools like MonkeyType is to produce type annotations from observed runtime behavior.

However, suppose you do want to create your own CLI tool with similar functionality as MonkeyType. In this blog post, we take a deeper dive and follow along the process to build on by developing our very own simplistic CLI tool that listens for function calls, analyzes associated functions with types together with generating type hints when necessary.

Learn how to construct a custom command-line interface (CLI) from scratch in this comprehensive tutorial. Designed for beginners, this guide will help you get started with Python programming and build a working tool step by step. 

Understanding the Basics

Now that you are getting ready to create your own, we will touch on some basic concepts so it makes sense.

1. Introduction To Type Annotations in Python : You can use type annotations to indicate the expected data types of variables, function arguments and return values in python. They are not enforced in runtime but still very useful for static analysis tools, IDEs and making the code cleaner to read.

2. Dynamic monkeytype Collection : This does not happen at compile time; instead, the compiler will observe during runtime that a variable has been assigned an integer value and so from this point forward it can be treated as such. That's the basic idea behind tools like MonkeyType

3. Introspection in Python : Very strong feature (in the sense ) which is ability to examine live objects at runtime functions and methods. This allows types of variables and function arguments to be tracked as they are used throughout execution.

Prerequisites

Before we dive in, make sure you have the following:

• Python 3.7+ installed on your machine.

• Basic understanding of Python's type annotations.

• Familiarity with Python's introspection capabilities (inspect module).

• pip for installing necessary packages.

Designing the CLI Tool

A DIY CLI version of MonkeyType would be designing a tool to do the following:

1. Track Function Calls: A tool should keep an eye on the function calls in your code, gathering details of what arguments you passed to each function and if it's not already obvious; what type is returned.

2. Storing Collected Data: After collecting the type information, it needs to be stored in a structured way. Type annotations will be generated from this data later on.

3. Generating Type Annotations: The tool should be trainable to generate these type annotations for the monitored functions based on data it collects. Then these annotations can be added to your source code.

4. Command-Line Interface (CLI) monkeytype : If it is hard to develop the tool, so no one will use this as such a basic utility, just create an intuitive and simple command line interface. With this interface, it lets the user define what modules to track and where is a good place to write type annotations.

Step 1: Project Setup

Begin by making a new directory for your CLI tool. Create a virtual environment in the directory and activate it as follows:

mkdir my_monkeytype_cli
cd my_monkeytype_cli
python3 -m venv venv
source venv/bin/activate  # On Windows, use `venv\Scripts\activate`

Next, install the required dependencies:

pip install click typeshed-client

• Click: A package for creating command line interfaces

• A type checking tool for working with Python type annotations.

Step 2 - Trace function calls and signatures

First create a dumb way to keep track of the function calls and argument types at runtime. Create new file as tracker py:

import inspect
from collections import defaultdict
from typing import Any, Callable, Dict, Tuple

class TypeTracker:
    def __init__(self):
        self.call_data = defaultdict(list)

    def track(self, func: Callable) -> Callable:
        def wrapper(*args, **kwargs):
            arg_types = tuple(type(arg).__name__ for arg in args)
            kwarg_types = {k: type(v).__name__ for k, v in kwargs.items()}
            return_type = type(func(*args, **kwargs)).__name__

            self.call_data[func.__name__].append((arg_types, kwarg_types, return_type))
            return func(*args, **kwargs)

        return wrapper

    def get_type_hints(self, func_name: str) -> str:
        func_data = self.call_data.get(func_name)
        if not func_data:
            return f"No data collected for function: {func_name}"

        arg_types, kwarg_types, return_type = func_data[0]
        args_hint = ', '.join(arg_types)
        kwargs_hint = ', '.join(f'{k}: {v}' for k, v in kwarg_types.items())
        return f"def {func_name}({args_hint}, {kwargs_hint}) -> {return_type}:"

Step 3: Building the CLI monkeytype

With the tracker in place, time to add a CLI using click. Create a new file named cli. Py:

import click
from tracker import TypeTracker

type_tracker = TypeTracker()

@click.group()
def cli():
    pass

@click.command()
@click.argument('function_name')
def annotate(function_name):
    """Generate type annotations for a given function."""
    type_hints = type_tracker.get_type_hints(function_name)
    click.echo(type_hints)

@click.command()
@click.argument('module_name')
def run(module_name):
    """Run the specified Python module with tracking enabled."""
    module = __import__(module_name)
    for name, func in inspect.getmembers(module, inspect.isfunction):
        setattr(module, name, type_tracker.track(func))
    click.echo(f"Tracking enabled for module: {module_name}")

cli.add_command(annotate)
cli.add_command(run)

if __name__ == "__main__":
    cli()

Step 4: Testing the CLI

Create a sample module of python named as example which helps to test you CLI. py:

def add(x, y):
    return x + y

def greet(name, times=1):
    return f"Hello, {name}!" * times

You may now use your CLI to monitor the function calls in this module.

python cli.py run example
python -c "import example; example.add(1, 2); example.greet('Alice')"
python cli.py annotate add
python cli.py annotate greet

You should see output like:

def add(int, int) -> int:
def greet(str, times=1) -> str:

Step 5: Improving the Tool

This is a simple version of the tool. Some things to change include:

1. Handle Complex Types : Extend the tracker to work with more complex types like lists, dictionaries and custom objects.

2. Persistence: Save the collected type information to a file and load it later for generating type annotations.

3. Automatic Annotation: Write the generated type hints directly into the source code files.

Key Components of the Tool

There are four main areas you need to cover, in order to create a fully functional CLI tool.

1. Tracking Function Wrapper: The core of the tool consisted in a function to wrap around another one, and great consequences that come from this operation. One approach is to do it by writing a function wrapper, that prints out the types of arguments and return values whenever this function gets called.

2. Data Encapsulation:- Type stored in a strong data structure. That is, something like a dictionary that connects function names with the arguments and return type of those functions.

3. Type Annotation Generation: The tool should know how to convert the type information it collected into Python types annotations. This involves reading and converting the stored data into Python syntax of type hinting.

4. User-Friendly CLI: By using a library like Click, you can make the tool interact with users without any effort. There should be commands in the CLI to get started with tracking a module, generate type annotations and maybe even auto-apply those for you —end of every one-hour pair-trading.session.

Enhancing the Tool

How to extend your working CLI tool from here once the basic functionality is in place

1. Support for Nested Data types: Expand the functionality of the tool to be able to deal with data structures that can store other nested complex objects :lists, dictionaries; custom Class Objects. This should make the type annotations more accurate and inclusive.

2. Persistence: In this resource we will include develop a functionality to save type information collected until some point into storage. This permits users to collect type data across multiple runs and emit annotations at a later time.

3. IDE Integration: An idea would be to integrate the tool with popular Python IDEs in order to give users a real-time type annotations suggestions based on their data.

4. Automated Code Modification: A good step ahead can be to ask the tool in further automatically insert these type annotations into source code for ConcatUser, ease user-file process.

Conclusion

Creating a CLI version of MonkeyType is also a great opportunity to learn how Python's type system works, as well its rich API for introspection and writing command-line tools. Even though this post is just a 500-foot view, the experience of actually writing out the program yourself can be valuable practice in Python and software development.

This will enable you to transform it into a robust tool that not only helps keep your code quality in-check, but also reduces the time and effort required when maintaining large Python codebase.