Added support for Imagen 3 in the Python and Node SDKs

Opper now support two image generation models, azure/dall-e-3-eu and gcp/imagen-3.0-generate-001-eu. Here is an example of generating an image from a description in Python:

def generate_image(description: str) -> ImageOutput:
    image, _ = opper.call(
        name="generate_image",
        output_type=ImageOutput,
        input=description,
        model="gcp/imagen-3.0-generate-001-eu",
        configuration=CallConfiguration(
            model_parameters={
                "aspectRatio": "9:16",
            }
        ),
    ) 
    return image


description = "portrait of a person standing in front of a park. vibrant, autumn colors"

path = save_file(generate_image(description).bytes)
print(path)

Here is a similar example in TypeScript:

async function testImageGeneration() {
    const image = await client.generateImage({
        model: "gcp/imagen-3.0-generate-001-eu",
        prompt: "portrait of a person standing in front of a park. vibrant, autumn colors",
        parameters: {
            aspectRatio: "9:16",
        }
    });

    const tempFilePath = path.join(os.tmpdir(), "image.png");
    fs.writeFileSync(tempFilePath, image.bytes);
    console.log(`image written to temporary file: ${tempFilePath}`);
}

testImageGeneration();

Model parameters vary between models, but here are the supported ones for each model:

azure/dall-e-3-eu:

  • style: natural, vivid
  • quality: standard, hd
  • size: 1024x1024, 1792x1024, 1024x1792

gcp/imagen-3.0-generate-001-eu:

  • aspectRatio: 1:1, 3:4, 4:3, 16:9, 9:16

Images as input to multimodal models

You are now able to pass images as input to multimodal models.

Python SDK

# special type for images, this is to capture the need for encoding the image in the right format
from opperai import ImageInput 

description, response = await aopper.call(
    name="async_describe_image",
    instructions="Create a short description of the image",
    output_type=Description,
    input=Image(
        image=ImageInput.from_path("examples/cat.png"),
    ),
    model="openai/gpt-4o",
)

Node SDK

// special function to read images, this is to capture the need for encoding the image in the right format
import { opperImage } from "opperai"; 

const { message } = await client.call({
    parent_span_uuid: trace.uuid,
    name: "node-sdk/call/multimodal/image-input",
    instructions: "Create a short description of the image",
    input: {image: image("examples/cat.png")},
    model: "openai/gpt-4o",
});

Image generation using DALL-E 3 now available

Using the ImageOutput type you are now able to generate images via call using DALL-E 3 in the Python SDK.

from opperai import ImageOutput

cat, _ = await aopper.call(
    name="generate_cat",
    output_type=ImageOutput,
    input="Create an image of a cat",
)

Using the Node SDK you can generate images using DALL-E 3.

const cat = await client.generateImage({
    parent_span_uuid: trace.uuid,
    prompt: "Create an image of a cat",
});

New models added

  • aws/claude-3.5-sonnet-eu
  • cerebras/llama3.1-8b
  • cerebras/llama3.1-70b
  • gcp/gemini-1.5-pro-002-eu
  • gcp/gemini-1.5-flash-002-eu
  • groq/llama-3.1-70b-versatile
  • groq/llama-3.1-8b-instant
  • groq/gemma2-9b-it
  • mistral/pixtral-12b-2409-eu
  • openai/o1-preview
  • openai/o1-mini

See Cerebras for more information about these models.

Updated default embedding model

The new default embedding model for indexes is text-embedding-3-large.

New models added

  • azure/meta-llama-3.1-405b
  • azure/meta-llama-3.1-70b-eu
  • azure/mistral-large-2407
  • mistral/mistral-large-2407
  • openai/gpt-4o-2024-05-13 (openai/gpt-4o currently points to this)
  • openai/gpt-4o-2024-08-06

Add examples at call time

You can now add examples at call time. This is useful if you have a set of examples that you want to use as a reference for your model without having to manage a dataset.

output, _ = opper.call(
    name="changelog/python/call-with-examples",
    instructions="extract the weekday from a text",
    examples=[
        Example(input="Today is Monday", output="Monday"),
        Example(input="Friday is the best day of the week", output="Friday"),
        Example(
            input="Saturday is the second best day of the week", output="Saturday"
        ),
    ],
    input="Wonder what day it is on Sunday",
)

The three ways of tracing your code using the Python SDK

Manually

span = opper.traces.start_trace(name="my_function", input="Hello, world!")
# business logic here
span.end()

Using context manager

with opper.traces.start(name="my_function", input="Hello, world!") as span:
    # business logic here

Using the @trace decorator

@trace
def my_function(input: str) -> str:
    # business logic here

Call a LLM without explicitly creating a function using the Python SDK

You can now call a LLM without explicitly creating a function.

opper.call(name="anthropic/claude-3-haiku", input="Hello, world!")

Manually trace using the Node SDK

You can now manually trace using the Node SDK.

// Start parent trace
const trace = await client.traces.start({
    name: "node-sdk/tracing-manual",
    input: "Trace initialization",
});

// You can optionally start a child span and provide the input
const span = await trace.startSpan({
    name: "node-sdk/tracing-manual/span",
    input: "Some input given to the span",
});

// A metric and/or comment can be saved to the span
// A span generation can also be saved using .saveGeneration()
await span.saveMetric({
    dimension: "accuracy",
    score: 1,
    comment: "This is a comment",
});

// End the span and provide the output
await span.end({
    output: JSON.stringify({ foo: "bar" }),
});

// End the parent trace
await trace.end({ output: JSON.stringify({ foo: "bar" }) });

Call a LLM without explicitly creating a function using the Node SDK

You can now call a LLM without explicitly creating a function.

const { message } = await client.call({
    name: "node-sdk/call/basic",
    input: "what is the capital of sweden",
});

Manually adding generations

You can now manually add generations to your traces. This is useful if you call an LLM outside of Opper but still want to use the tracing capabilities of Opper.

def run():
    opper = Opper()
    spans = opper.spans

    with spans.start("transform", input="Hello, world!") as span:
        t0 = datetime.now(timezone.utc)
        manually_call_llm()
        t1 = datetime.now(timezone.utc)
        span.save_generation(
            called_at=t0,
            duration_ms=int((t1 - t0).total_seconds() * 1000),
            response="I'm happy because I'm happy",
            model="anthropic/claude-3-haiku",
            messages=[
                {
                    "role": "user",
                    "content": "Hello, world!",
                }
            ],
            cost=3.1,
            prompt_tokens=10,
            completion_tokens=10,
            total_tokens=20,
        )

OpenAI model GPT-4o mini now available

We have added support for the just released GPT-4o mini model from OpenAI.

Projects now available

Projects

Projects allow you to create separation in Opper. Currently, the following is tied to a project:

  • Functions
  • Indexes
  • Traces
  • API keys

When you create an API for a specific project, all usage will be associated with that specific project automatically, so there is no need to pass the project as you are using it.

Manage organizations and invite your colleagues

Projects

You are now able to create your own organizations in Opper. Go to Settings --> Organization and click Create Organization in the top right corner to get started.

Once you have created your organization, you are able to invite your colleagues by sending an invite to their email address. Once they are in, you are able to collaborate and have a common view of your AI usage.