SCIP

Simple Conversation and Interaction Protocol for LLM.

sequenceDiagram
    participant C as Client
    participant S as Gate
    participant LLM as 3rd Party LLM Service

    C->>S: Send query with conversation history
    S->>LLM: Forward query

    loop Process streaming chunks
        %% Streaming response from LLM indicated with a double‐equal arrow
        LLM->>S: Stream response chunk or tool use directive
        alt Error in streaming response
            LLM->>S: Stream error message
            S->>C: Forward error to client
            Note right of S: End streaming chunks loop
        else Tool use detected
            loop Tool use chain
                alt Server tool use
                    S->>S: Execute server-side tool
                    alt Error executing server-side tool
                        S->>S: Handle error (log, retry, etc.)
                        S->>LLM: Report tool error
                        LLM->>S: Stream error follow-up directive
                    end
                else Client tool use (forwarded from server)
                    S->>C: Forward tool use request
                    C->>C: Execute client-side tool
                    alt Error executing client-side tool
                        C->>S: Return error result
                        S->>LLM: Report tool error
                        LLM->>S: Stream error follow-up directive
                    else No error
                        C->>S: Return tool use result
                    end
                end
                S->>LLM: Update context with tool result (or error)
                LLM->>S: Stream follow-up directive or response chunk
                alt Error in follow-up directive
                    LLM->>S: Stream error follow-up message
                    S->>C: Forward error to client
                    Note right of S: End tool chain loop
                else No further tool use
                    Note right of S: End tool chain loop
                end
            end
        else Normal response chunk
            S->>C: Stream response chunk
        end
    end