Google Gemini Conversation Format (Generate Content)

Google Gemini Conversation Format (Generate Content)

Page Overview

“Official Documentation” Google Gemini Generating content API

📝 Introduction

The Google Gemini API supports generating content using images, audio, code, tools, and more. Given a GenerateContentRequest, it returns a model response. It supports text generation, vision understanding, audio processing, long context, code execution, JSON mode, function calling, and more.

💡 Request Examples

Basic Text Chat ✅

curl "https://your-4All API-server-address/v1beta/models/gemini-2.0-flash:generateContent?key=$4All API_API_KEY" \
    -H 'Content-Type: application/json' \
    -X POST \
    -d '{
      "contents": [{
        "parts":[{"text": "Write a story about a magic backpack."}]
        }]
       }' 2> /dev/null

Image Analysis Chat ✅

# Use a temporary file to store the base64-encoded image data
TEMP_B64=$(mktemp)
trap 'rm -f "$TEMP_B64"' EXIT
base64 $B64FLAGS $IMG_PATH > "$TEMP_B64"

# Use a temporary file to store the JSON payload
TEMP_JSON=$(mktemp)
trap 'rm -f "$TEMP_JSON"' EXIT

cat > "$TEMP_JSON" << EOF
{
  "contents": [{
    "parts":[
      {"text": "Tell me about this instrument"},
      {
        "inline_data": {
          "mime_type":"image/jpeg",
          "data": "$(cat "$TEMP_B64")"
        }
      }
    ]
  }]
}
EOF

curl "https://your-4All API-server-address/v1beta/models/gemini-2.0-flash:generateContent?key=$4All API_API_KEY" \
    -H 'Content-Type: application/json' \
    -X POST \
    -d "@$TEMP_JSON" 2> /dev/null

Function Calling ✅

cat > tools.json << EOF
{
  "function_declarations": [
    {
      "name": "enable_lights",
      "description": "Turn on the lighting system."
    },
    {
      "name": "set_light_color",
      "description": "Set the light color. Lights must be enabled for this to work.",
      "parameters": {
        "type": "object",
        "properties": {
          "rgb_hex": {
            "type": "string",
            "description": "The light color as a 6-digit hex string, e.g. ff0000 for red."
          }
        },
        "required": [
          "rgb_hex"
        ]
      }
    },
    {
      "name": "stop_lights",
      "description": "Turn off the lighting system."
    }
  ]
}
EOF

curl "https://your-4All API-server-address/v1beta/models/gemini-2.0-flash:generateContent?key=$4All API_API_KEY" \
  -H 'Content-Type: application/json' \
  -d @<(echo '
  {
    "system_instruction": {
      "parts": {
        "text": "You are a helpful lighting system bot. You can turn lights on and off, and you can set the color. Do not perform any other tasks."
      }
    },
    "tools": ['$(cat tools.json)'],

    "tool_config": {
      "function_calling_config": {"mode": "auto"}
    },

    "contents": {
      "role": "user",
      "parts": {
        "text": "Turn on the lights please."
      }
    }
  }
') 2>/dev/null |sed -n '/"content"/,/"finishReason"/p'

JSON Mode Response ✅

curl "https://your-4All API-server-address/v1beta/models/gemini-2.0-flash:generateContent?key=$4All API_API_KEY" \
-H 'Content-Type: application/json' \
-d '{
    "contents": [{
      "parts":[
        {"text": "List 5 popular cookie recipes"}
        ]
    }],
    "generationConfig": {
        "response_mime_type": "application/json",
        "response_schema": {
          "type": "ARRAY",
          "items": {
            "type": "OBJECT",
            "properties": {
              "recipe_name": {"type":"STRING"},
            }
          }
        }
    }
}' 2> /dev/null | head

Audio Processing 🟡

!!! warning “File upload restriction” Only base64 uploads via inline_data are supported for audio. file_data.file_uri and the File API are not supported.

# Upload audio data to the API request using the File API
# Upload audio data to the API request using base64 inline_data
if [[ "$(base64 --version 2>&1)" = *"FreeBSD"* ]]; then
  B64FLAGS="--input"
else
  B64FLAGS="-w0"
fi
AUDIO_B64=$(base64 $B64FLAGS "$AUDIO_PATH")

curl "https://your-4All API-server-address/v1beta/models/gemini-2.0-flash:generateContent?key=$4All API_API_KEY" \
  -H 'Content-Type: application/json' \
  -X POST \
  -d '{
    "contents": [{
      "parts": [
        {"text": "Please describe this audio file."},
        {"inline_data": {"mime_type": "audio/mpeg", "data": "'$AUDIO_B64'"}}
      ]
    }]
  }' 2> /dev/null | jq ".candidates[].content.parts[].text"

Video Processing 🟡

!!! warning “File upload restriction” Only base64 uploads via inline_data are supported for video. file_data.file_uri and the File API are not supported.

# Upload video data to the API request using the File API
# Upload video data to the API request using base64 inline_data
if [[ "$(base64 --version 2>&1)" = *"FreeBSD"* ]]; then
  B64FLAGS="--input"
else
  B64FLAGS="-w0"
fi
VIDEO_B64=$(base64 $B64FLAGS "$VIDEO_PATH")

curl "https://your-4All API-server-address/v1beta/models/gemini-2.0-flash:generateContent?key=$4All API_API_KEY" \
  -H 'Content-Type: application/json' \
  -X POST \
  -d '{
    "contents": [{
      "parts": [
        {"text": "Transcribe the audio from this video and provide visual descriptions."},
        {"inline_data": {"mime_type": "video/mp4", "data": "'$VIDEO_B64'"}}
      ]
    }]
  }' 2> /dev/null | jq ".candidates[].content.parts[].text"

PDF Processing 🟡

!!! warning “File upload restriction” Only base64 uploads via inline_data are supported for PDFs. file_data.file_uri and the File API are not supported.

MIME_TYPE=$(file -b --mime-type "${PDF_PATH}")
# Upload PDF files to the API request using base64 inline_data
if [[ "$(base64 --version 2>&1)" = *"FreeBSD"* ]]; then
  B64FLAGS="--input"
else
  B64FLAGS="-w0"
fi
PDF_B64=$(base64 $B64FLAGS "$PDF_PATH")

echo $MIME_TYPE

curl "https://your-4All API-server-address/v1beta/models/gemini-2.0-flash:generateContent?key=$4All API_API_KEY" \
  -H 'Content-Type: application/json' \
  -X POST \
  -d '{
    "contents": [{
      "parts": [
        {"text": "Can you add a few more lines to this poem?"},
        {"inline_data": {"mime_type": "application/pdf", "data": "'$PDF_B64'"}}
      ]
    }]
  }' 2> /dev/null | jq ".candidates[].content.parts[].text"

Chat Conversation ✅

curl https://your-4All API-server-address/v1beta/models/gemini-2.0-flash:generateContent?key=$4All API_API_KEY \
    -H 'Content-Type: application/json' \
    -X POST \
    -d '{
      "contents": [
        {"role":"user",
         "parts":[{
           "text": "Hello"}]},
        {"role": "model",
         "parts":[{
           "text": "Great to meet you. What would you like to know?"}]},
        {"role":"user",
         "parts":[{
           "text": "I have two dogs in my house. How many paws are in my house?"}]},
      ]
    }' 2> /dev/null | grep "text"

Streaming Response ✅

curl "https://your-4All API-server-address/v1beta/models/gemini-2.0-flash:streamGenerateContent?alt=sse&key=$4All API_API_KEY" \
    -H 'Content-Type: application/json' \
    --no-buffer \
    -d '{
      "contents": [{
        "parts": [{"text": "Write a story about a magic backpack"}]
      }]
    }'

Code Execution ✅

curl "https://your-4All API-server-address/v1beta/models/gemini-2.0-flash:generateContent?key=$4All API_API_KEY" \
    -H 'Content-Type: application/json' \
    -X POST \
    -d '{
      "contents": [{
        "parts": [{"text": "Calculate the 10th number in the Fibonacci sequence"}]
      }],
      "tools": [{
        "codeExecution": {}
      }]
    }'

Generation Config ✅

curl https://your-4All API-server-address/v1beta/models/gemini-2.0-flash:generateContent?key=$4All API_API_KEY \
    -H 'Content-Type: application/json' \
    -X POST \
    -d '{
        "contents": [{
            "parts":[
                {"text": "Explain how AI works"}
            ]
        }],
        "generationConfig": {
            "stopSequences": [
                "Title"
            ],
            "temperature": 1.0,
            "maxOutputTokens": 800,
            "topP": 0.8,
            "topK": 10
        }
    }'  2> /dev/null | grep "text"

Safety Settings ✅

echo '{
    "safetySettings": [
        {"category": "HARM_CATEGORY_HARASSMENT", "threshold": "BLOCK_ONLY_HIGH"},
        {"category": "HARM_CATEGORY_HATE_SPEECH", "threshold": "BLOCK_MEDIUM_AND_ABOVE"}
    ],
    "contents": [{
        "parts":[{
            "text": "'I support Martians Soccer Club and I think Jupiterians Football Club sucks! Write a ironic phrase about them.'"}]}]}' > request.json

curl "https://your-4All API-server-address/v1beta/models/gemini-2.0-flash:generateContent?key=$4All API_API_KEY" \
    -H 'Content-Type: application/json' \
    -X POST \
    -d @request.json 2> /dev/null

System Instruction ✅

curl "https://your-4All API-server-address/v1beta/models/gemini-2.0-flash:generateContent?key=$4All API_API_KEY" \
-H 'Content-Type: application/json' \
-d '{ "system_instruction": {
    "parts":
      { "text": "You are a cat. Your name is Neko."}},
    "contents": {
      "parts": {
        "text": "Hello there"}}}'

📮 Request

Endpoints

Generate content

POST https://your-4All API-server-address/v1beta/{model=models/*}:generateContent

Stream generate content

POST https://your-4All API-server-address/v1beta/{model=models/*}:streamGenerateContent

Authentication Method

Include the API key in the request URL parameters:

?key=$4All API_API_KEY

Where $4All API_API_KEY is your Google AI API key.

Path Parameters

model

• Type: string
• Required: Yes

The name of the model used to generate the completion.

Format: models/{model}, for example models/gemini-2.0-flash

Request Body Parameters

contents

• Type: array
• Required: Yes

The conversation content with the model. For a single-turn query, this is a single instance. For multi-turn queries such as chat, this is a repeated field containing the conversation history and the latest request.

Content object properties:

Property	Type	Required	Description
parts	array	Yes	Ordered content parts that make up a single message
role	string	No	The producer of the content in the conversation. user, model, function, or tool

Part object properties:

Property	Type	Required	Description
text	string	No	Plain text content
inlineData	object	No	Inline media byte data
fileData	object	No	URI reference to an uploaded file
functionCall	object	No	Function call request
functionResponse	object	No	Function call response
executableCode	object	No	Executable code
codeExecutionResult	object	No	Code execution result

InlineData object properties:

Property	Type	Required	Description
mimeType	string	Yes	The media MIME type
data	string	Yes	Base64-encoded media data

FileData object properties:

Property	Type	Required	Description
mimeType	string	Yes	The file MIME type
fileUri	string	Yes	The file URI

tools

• Type: array
• Required: No

A list of tools that the model may use to generate the next response. Supported tools include function calling and code execution.

Tool object properties:

Property	Type	Required	Description
functionDeclarations	array	No	An optional list of function declarations
codeExecution	object	No	Enables the model to execute code

FunctionDeclaration object properties:

Property	Type	Required	Description
name	string	Yes	Function name
description	string	No	Description of what the function does
parameters	object	No	Function parameters in JSON Schema format

FunctionCall object properties:

Property	Type	Required	Description
name	string	Yes	The name of the function to call
args	object	No	Key-value pairs of function arguments

FunctionResponse object properties:

Property	Type	Required	Description
name	string	Yes	The name of the function called
response	object	Yes	Response data from the function call

ExecutableCode object properties:

Property	Type	Required	Description
language	enum	Yes	The programming language of the code
code	string	Yes	The code to execute

CodeExecutionResult object properties:

Property	Type	Required	Description
outcome	enum	Yes	The status of the code execution result
output	string	No	Output generated by the code execution

CodeExecution object properties:

Property	Type	Required	Description
{}	empty object	-	Empty configuration object to enable code execution

toolConfig

• Type: object
• Required: No

Tool configuration for any tools specified in the request.

ToolConfig object properties:

Property	Type	Required	Description
functionCallingConfig	object	No	Function calling configuration

FunctionCallingConfig object properties:

Property	Type	Required	Description
mode	enum	No	Specifies the function calling mode
allowedFunctionNames	array	No	A list of function names allowed to be called

FunctionCallingMode enum values:

• MODE_UNSPECIFIED : Default mode; the model decides whether to call a function
• AUTO : The model automatically decides when to call functions
• ANY : The model must call a function
• NONE : The model cannot call functions

safetySettings

• Type: array
• Required: No

A list of SafetySetting instances used to block unsafe content.

SafetySetting object properties:

Property	Type	Required	Description
category	enum	Yes	Safety category
threshold	enum	Yes	Blocking threshold

HarmCategory enum values:

• HARM_CATEGORY_HARASSMENT : Harassment content
• HARM_CATEGORY_HATE_SPEECH : Hate speech and hateful content
• HARM_CATEGORY_SEXUALLY_EXPLICIT : Sexually explicit content
• HARM_CATEGORY_DANGEROUS_CONTENT : Dangerous content
• HARM_CATEGORY_CIVIC_INTEGRITY : Content that may be used to undermine civic integrity

HarmBlockThreshold enum values:

• BLOCK_LOW_AND_ABOVE : Allows content rated NEGLIGIBLE
• BLOCK_MEDIUM_AND_ABOVE : Allows content rated NEGLIGIBLE and LOW
• BLOCK_ONLY_HIGH : Allows content rated NEGLIGIBLE, LOW, and MEDIUM
• BLOCK_NONE : Allows all content
• OFF : Turns off the safety filter

HarmBlockThreshold full enum values:

• HARM_BLOCK_THRESHOLD_UNSPECIFIED : Threshold not specified
• BLOCK_LOW_AND_ABOVE : Blocks harmful content with low probability and above; only NEGLIGIBLE content is allowed
• BLOCK_MEDIUM_AND_ABOVE : Blocks harmful content with medium probability and above; allows NEGLIGIBLE and LOW content
• BLOCK_ONLY_HIGH : Blocks only harmful content with high probability; allows NEGLIGIBLE, LOW, and MEDIUM content
• BLOCK_NONE : Blocks no content; allows all content levels
• OFF : Completely turns off the safety filter

systemInstruction

• Type: object (Content)
• Required: No

Developer-defined system instruction. Currently only text is supported.

generationConfig

• Type: object
• Required: No

Configuration options for model generation and output.

GenerationConfig object properties:

Property	Type	Required	Description
stopSequences	array	No	A set of character sequences that stop generation output, up to 5