Deterministic Interface Discovery Schema (DIDS)

The Deterministic Interface Discovery Schema (DIDS) enables contributors to define framework and library patterns for Unoplat Code Confluence to detect and analyze across different programming languages. This schema provides a structured way to describe how frameworks and libraries should be identified in codebases.

Background and Problem Statement

Background

Modern code intelligence systems usually cannot read entire repositories for every request. In large codebases, they typically rely on a retrieval pipeline:

1. Chunk source files into indexable units
2. Compute embeddings for semantic lookup
3. Retrieve nearest candidates from the index
4. Rerank candidates for relevance
5. Assemble final context for analysis or generation

This pattern works well for broad navigation and question answering, but it is fundamentally probabilistic. Retrieval quality depends on chunking boundaries, embedding behavior, reranking heuristics, and prompt phrasing.

At scale, this pipeline is also expensive to execute and maintain: indexing/embedding jobs must be recomputed, vector infrastructure must be operated, and retrieval quality must be continuously tuned. The approach is also non-extensible for deterministic interface discovery because framework-specific logic is usually implemented as ad-hoc runtime behavior instead of a reusable contract.

For interface discovery, we need stronger guarantees than "likely relevant context." We need reproducible detection of framework and library constructs so downstream systems can build stable blueprints, generate reliable structured output, and reason across repositories consistently.

The Deterministic Interface Discovery Schema (DIDS) introduces that missing contract: a versioned, machine-enforceable way to describe what to detect and how those detections should be interpreted.

Problem Statement

For any coding task, an agent or human needs a reliable blueprint of the codebase. Today, that blueprint is inferred mostly from semantic retrieval and ad-hoc heuristics, which creates following problems:

1. Non-determinism: the same repository can produce different detections across runs.
2. High execution and maintenance cost: embeddings, indexes, and tuning loops create ongoing operational overhead.
3. Low extensibility: adding support for new frameworks or internal patterns requires custom, scattered logic.
4. Weak verifiability: behavior is hard to validate in CI because rules are implicit instead of contract-driven.
5. Lack of open standards: without an open, versioned schema, definitions are harder to share, interoperate, and evolve safely.

We therefore need a deterministic, schema-first standard that defines framework and library features with explicit constraints and validation rules, enabling consistent interface discovery across any tech ecosystem.

Goals

This specification work aims to:

1. Define a machine-enforceable detection contract for framework/library features based on programming constructs like annotations, calls, imports etc.
2. Improve contributor correctness with explicit constraints and validation expectations
3. Support safe schema evolution with clear compatibility and migration expectations

Schema Structure

Top Level Structure

The schema organizes definitions hierarchically by language and library:

{
  "python": {
    "fastapi": { /* library definition */ },
    "django": { /* library definition */ }
  },
  "javascript": {
    "express": { /* library definition */ },
    "react": { /* library definition */ }
  }
}

Supported Languages

Frameworks Supported

Browse the Supported Frameworks catalog to see currently available libraries by programming language.

Library Definition Structure

Each library definition describes a framework or library that Unoplat Code Confluence should recognize.

Required Fields

Optional Fields

Feature Definition Structure

Each feature within a library describes a specific pattern that Unoplat Code Confluence should detect.

Required Fields

target_level Values

concept Values

Detection Heuristics (v3)

Detection is import-gated and regex-based. absolute_paths must be present in the file imports, and construct_query refines the tree-sitter regex patterns used to match decorators, calls, inheritance, and exported function declarations.

Optional Fields

construct_query Properties

Real Framework Examples

Explore actual schema definitions from the Unoplat Code Confluence codebase:

{
  "python": {
    "fastapi": {
      "docs_url": "https://fastapi.tiangolo.com",
      "features": {
        "http_endpoint": {
          "description": "HTTP verb decorator that registers a route handler",
          "absolute_paths": [
            "fastapi.FastAPI",
            "fastapi.applications.FastAPI"
          ],
          "target_level": "function",
          "concept": "AnnotationLike",
          "construct_query": {
            "method_regex": "(get|post|put|delete|patch|head|options|trace)"
          },
          "startpoint": true
        },
        "background_tasks": {
          "description": "Background task execution using BackgroundTasks",
          "absolute_paths": [
            "fastapi.BackgroundTasks"
          ],
          "target_level": "function",
          "concept": "CallExpression"
        },
        "api_router": {
          "description": "Route organization using APIRouter",
          "absolute_paths": [
            "fastapi.APIRouter"
          ],
          "target_level": "function",
          "concept": "AnnotationLike",
          "construct_query": {
            "method_regex": "(get|post|put|delete|patch|head|options|trace|include_router)"
          },
          "startpoint": true
        }
      }
    }
  }
}

{
  "python": {
    "pydantic": {
      "docs_url": "https://docs.pydantic.dev",
      "features": {
        "data_model": {
          "description": "Base class for Pydantic models with automatic validation",
          "absolute_paths": [
            "pydantic.BaseModel",
            "pydantic.main.BaseModel"
          ],
          "target_level": "class",
          "concept": "Inheritance"
        }
      }
    }
  }
}

{
  "python": {
    "sqlalchemy": {
      "docs_url": "https://docs.sqlalchemy.org",
      "features": {
        "declarative_base": {
          "description": "Base class for SQLAlchemy declarative models",
          "absolute_paths": [
            "sqlalchemy.ext.declarative.declarative_base",
            "sqlalchemy.orm.declarative_base"
          ],
          "target_level": "class",
          "concept": "Inheritance"
        },
        "mapped_column": {
          "description": "Column definition using mapped_column for modern SQLAlchemy",
          "absolute_paths": [
            "sqlalchemy.orm.mapped_column"
          ],
          "target_level": "function",
          "concept": "CallExpression"
        },
        "relationship": {
          "description": "Relationship definition between models",
          "absolute_paths": [
            "sqlalchemy.orm.relationship"
          ],
          "target_level": "function",
          "concept": "CallExpression"
        },
        "column": {
          "description": "Traditional column definition using Column",
          "absolute_paths": [
            "sqlalchemy.Column",
            "sqlalchemy.sql.schema.Column"
          ],
          "target_level": "function",
          "concept": "CallExpression"
        }
      }
    }
  }
}

{
  "python": {
    "sqlmodel": {
      "docs_url": "https://sqlmodel.tiangolo.com",
      "features": {
        "sqlmodel_base": {
          "description": "Base class for SQLModel models combining Pydantic and SQLAlchemy",
          "absolute_paths": [
            "sqlmodel.SQLModel"
          ],
          "target_level": "class",
          "concept": "Inheritance"
        },
        "field_definition": {
          "description": "Field definition using Field() function with SQLModel extensions",
          "absolute_paths": [
            "sqlmodel.Field"
          ],
          "target_level": "function",
          "concept": "CallExpression"
        },
        "relationship": {
          "description": "Relationship definition between SQLModel models",
          "absolute_paths": [
            "sqlmodel.Relationship"
          ],
          "target_level": "function",
          "concept": "CallExpression"
        }
      }
    }
  }
}

Complete Multi-Framework Example

Validation Rules

Pattern Validation

Best Practices

1. Comprehensive Coverage

• Define all major features of a framework/library
• Include both common and advanced usage patterns
• Cover different ways the same concept might be expressed

2. Accurate Paths

• Use fully-qualified import paths
• Test paths against actual library documentation
• Include version information when patterns change between versions

3. Clear Descriptions

• Write human-readable descriptions
• Explain the purpose and context of each feature
• Include examples in notes when helpful

4. Proper Categorization

• Choose appropriate concept values based on how the feature is used
• Use startpoint: true for entry points like API endpoints
• Set correct target_level based on where the feature applies

5. Strategic Regex Usage

• Use construct_query regex patterns to refine matches
• Test regex patterns against real code examples
• Keep patterns specific enough to avoid false positives

Contributing Schema Updates

To contribute new framework/library definitions:

1. Fork the repository and create a feature branch
2. Update the schema file at /public/schemas/custom-framework-lib-schema.json
3. Add comprehensive definitions following the structure above
4. Test your definitions against real codebases
5. Submit a pull request with detailed examples and test cases

Testing Your Schema

Before submitting, verify your schema definitions:

1. JSON Validation — Ensure the file is valid JSON
2. Schema Validation — Validate against the JSON Schema specification
3. Real-world Testing — Test against actual codebases using the framework
4. Documentation — Ensure all referenced URLs are accessible

Troubleshooting

Common Issues

Future Enhancements

The schema is designed to be extensible. Future versions may include:

• Additional programming languages (Rust, C#, Kotlin)
• More concept types for different framework constructs
• Enhanced query refinement options for complex patterns
• Version-aware detection for handling breaking changes between library versions