Skip to content

CLAUDE.md

Latest commit

 

History

History
523 lines (394 loc) · 14.3 KB

CLAUDE.md

File metadata and controls

523 lines (394 loc) · 14.3 KB

Sentry Development Guide for Claude

Overview

Sentry is a developer-first error tracking and performance monitoring platform. This repository contains the main Sentry application, which is a large-scale Django application with a React frontend.

Tech Stack

Frontend

See static/CLAUDE.md for frontend development guide.

Backend

  • Language: Python 3.13+
  • Framework: Django 5.2+
  • API: Django REST Framework with drf-spectacular for OpenAPI docs
  • Task Queue: Celery 5.5+
  • Databases: PostgreSQL (primary), Redis, ClickHouse (via Snuba)
  • Message Queue: Kafka, RabbitMQ
  • Stream Processing: Arroyo (Kafka consumer/producer framework)
  • Cloud Services: Google Cloud Platform (Bigtable, Pub/Sub, Storage, KMS)

Infrastructure

  • Container: Docker (via devservices)
  • Package Management: pnpm (Node.js), pip (Python)
  • Node Version: 22 (managed by Volta)

Project Structure

sentry/
├── src/
│   ├── sentry/           # Main Django application
│   │   ├── api/          # REST API endpoints
│   │   ├── models/       # Django models
│   │   ├── tasks/        # Celery tasks
│   │   ├── integrations/ # Third-party integrations
│   │   ├── issues/       # Issue tracking logic
│   │   └── web/          # Web views and middleware
│   ├── sentry_plugins/   # Plugin system
│   └── social_auth/      # Social authentication
├── static/               # Frontend application (see static/CLAUDE.md)
├── tests/                # Test suite
├── fixtures/             # Test fixtures
├── devenv/               # Development environment config
├── migrations/           # Database migrations
└── config/               # Configuration files

Key Commands

Development Setup

# Install dependencies and setup development environment
make develop

# Or use the newer devenv command
devenv sync

# Start the development server
pnpm run dev

Testing

# Run Python tests
pytest

# Run specific test file
pytest tests/sentry/api/test_base.py

Code Quality

# Preferred: Run pre-commit hooks on specific files
pre-commit run --files src/sentry/path/to/file.py

# Run all pre-commit hooks
pre-commit run --all-files

# Individual linting tools (use pre-commit instead when possible)
black --check  # Run black first
isort --check
flake8

Database Operations

# Run migrations
sentry django migrate

# Create new migration
sentry django makemigrations

# Reset database
make reset-db

Development Services

Sentry uses devservices to manage local development dependencies:

  • PostgreSQL: Primary database
  • Redis: Caching and queuing
  • Snuba: ClickHouse-based event storage
  • Relay: Event ingestion service
  • Symbolicator: Debug symbol processing
  • Taskbroker: Asynchronous task processing
  • Spotlight: Local debugging tool

AI Assistant Quick Decision Trees

"User wants to add an API endpoint"

  1. Check if endpoint already exists: grep -r "endpoint_name" src/sentry/api/
  2. Inherit from appropriate base:
    • Organization-scoped: OrganizationEndpoint
    • Project-scoped: ProjectEndpoint
    • Region silo: RegionSiloEndpoint
  3. File locations:
    • Endpoint: src/sentry/api/endpoints/{resource}.py
    • URL: src/sentry/api/urls.py
    • Test: tests/sentry/api/endpoints/test_{resource}.py
    • Serializer: src/sentry/api/serializers/models/{model}.py

"User wants to add a Celery task"

  1. Location: src/sentry/tasks/{category}.py
  2. Use @instrumented_task decorator
  3. Set appropriate queue and max_retries
  4. Test location: tests/sentry/tasks/test_{category}.py

Critical Patterns (Copy-Paste Ready)

API Endpoint Pattern

# src/sentry/core/endpoints/organization_details.py
from rest_framework.request import Request
from rest_framework.response import Response
from sentry.api.base import region_silo_endpoint
from sentry.api.bases.organization import OrganizationEndpoint
from sentry.api.serializers import serialize
from sentry.api.serializers.models.organization import DetailedOrganizationSerializer

@region_silo_endpoint
class OrganizationDetailsEndpoint(OrganizationEndpoint):
    publish_status = {
        "GET": ApiPublishStatus.PUBLIC,
        "PUT": ApiPublishStatus.PUBLIC,
    }

    def get(self, request: Request, organization: Organization) -> Response:
        """Get organization details."""
        return Response(
            serialize(
                organization,
                request.user,
                DetailedOrganizationSerializer()
            )
        )

# Add to src/sentry/api/urls.py:
# path('organizations/<slug:organization_slug>/', OrganizationDetailsEndpoint.as_view()),

Celery Task Pattern

# src/sentry/tasks/email.py
from sentry.tasks.base import instrumented_task

@instrumented_task(
    name="sentry.tasks.send_email",
    queue="email",
    max_retries=3,
    default_retry_delay=60,
)
def send_email(user_id: int, subject: str, body: str) -> None:
    from sentry.models import User

    try:
        user = User.objects.get(id=user_id)
        # Send email logic
    except User.DoesNotExist:
        # Don't retry if user doesn't exist
        return

API Development

Adding New Endpoints

  1. Create endpoint in src/sentry/api/endpoints/
  2. Add URL pattern in src/sentry/api/urls.py
  3. Document with drf-spectacular decorators
  4. Add tests in tests/sentry/api/endpoints/

API Documentation

  • OpenAPI spec generation: make build-api-docs
  • API ownership tracked in src/sentry/apidocs/api_ownership_allowlist_dont_modify.py

API Design Rules

  1. Route: /api/0/organizations/{org}/projects/{project}/
  2. Use snake_case for URL params
  3. Use camelCase for request/response bodies
  4. Return strings for numeric IDs
  5. Implement pagination with cursor
  6. Use GET for read, POST for create, PUT for update

Testing Best Practices

Python Tests

  • Use pytest fixtures
  • Mock external services
  • Test database isolation with transactions
  • Use factories for test data
  • For Kafka/Arroyo components: Use LocalProducer with MemoryMessageStorage instead of mocks

Test Pattern

# tests/sentry/core/endpoints/test_organization_details.py
from sentry.testutils.cases import APITestCase

class OrganizationDetailsTest(APITestCase):
    endpoint = "sentry-api-0-organization-details"

    def test_get_organization(self):
        org = self.create_organization(owner=self.user)
        self.login_as(self.user)

        response = self.get_success_response(org.slug)
        assert response.data["id"] == str(org.id)

Common Patterns

Feature Flags

from sentry import features

if features.has('organizations:new-feature', organization):
    # New feature code

Permissions

from sentry.api.permissions import SentryPermission

class MyPermission(SentryPermission):
    scope_map = {
        'GET': ['org:read'],
        'POST': ['org:write'],
    }

Logging Pattern

import logging
from sentry import analytics

logger = logging.getLogger(__name__)

# Structured logging
logger.info(
    "user.action.complete",
    extra={
        "user_id": user.id,
        "action": "login",
        "ip_address": request.META.get("REMOTE_ADDR"),
    }
)

# Analytics event
analytics.record(
    "feature.used",
    user_id=user.id,
    organization_id=org.id,
    feature="new-dashboard",
)

Arroyo Stream Processing

# Using Arroyo for Kafka producers with dependency injection for testing
from arroyo.backends.abstract import Producer
from arroyo.backends.kafka import KafkaProducer, KafkaPayload
from arroyo.backends.local.backend import LocalBroker
from arroyo.backends.local.storages.memory import MemoryMessageStorage

# Production producer
def create_kafka_producer(config):
    return KafkaProducer(build_kafka_configuration(default_config=config))

# Test producer using Arroyo's LocalProducer
def create_test_producer_factory():
    storage = MemoryMessageStorage()
    broker = LocalBroker(storage)
    return lambda config: broker.get_producer(), storage

# Dependency injection pattern for testable Kafka producers
class MultiProducer:
    def __init__(self, topic: Topic, producer_factory: Callable[[Mapping[str, object]], Producer[KafkaPayload]] | None = None):
        self.producer_factory = producer_factory or self._default_producer_factory
        # ... setup code

    def _default_producer_factory(self, config) -> KafkaProducer:
        return KafkaProducer(build_kafka_configuration(default_config=config))

Architecture Rules

Silo Mode

  • Control Silo: User auth, billing, organization management
  • Region Silo: Project data, events, issues
  • Check model's silo in src/sentry/models/outbox.py
  • Use @region_silo_endpoint or @control_silo_endpoint

Database Guidelines

  1. NEVER join across silos
  2. Use outbox for cross-silo updates
  3. Migrations must be backwards compatible
  4. Add indexes for queries on 1M+ row tables
  5. Use db_index=True or db_index_together

Anti-Patterns (NEVER DO)

Backend

# WRONG: Direct model import in API
from sentry.models import Organization  # NO!

# RIGHT: Use endpoint bases
from sentry.api.bases.organization import OrganizationEndpoint

# WRONG: Synchronous external calls
response = requests.get(url)  # NO!

# RIGHT: Use Celery task
from sentry.tasks import fetch_external_data
fetch_external_data.delay(url)

# WRONG: N+1 queries
for org in organizations:
    org.projects.all()  # NO!

# RIGHT: Use prefetch_related
organizations.prefetch_related('projects')

# WRONG: Use hasattr() for unions
x: str | None = "hello"
if hasattr(x, "replace"):
    x = x.replace("e", "a")

# RIGHT: Use isinstance()
x: str | None = "hello"
if isinstance(x, str):
    x = x.replace("e", "a")

Performance Considerations

  1. Use database indexing appropriately
  2. Implement pagination for list endpoints
  3. Cache expensive computations with Redis
  4. Use Celery for background tasks
  5. Optimize queries with select_related and prefetch_related

Security Guidelines

  1. Always validate user input
  2. Use Django's CSRF protection
  3. Implement proper permission checks
  4. Sanitize data before rendering
  5. Follow OWASP guidelines

Debugging Tips

  1. Use sentry devserver for full stack debugging
  2. Access Django shell: sentry django shell
  3. View Celery tasks: monitor RabbitMQ management UI
  4. Database queries: use Django Debug Toolbar

Quick Debugging

# Print SQL queries
from django.db import connection
print(connection.queries)

# Debug Celery task
from sentry.tasks import my_task
my_task.apply(args=[...]).get()  # Run synchronously

# Check feature flag
from sentry import features
features.has('organizations:feature', org)

# Current silo mode
from sentry.silo import SiloMode
from sentry.services.hybrid_cloud import silo_mode_delegation
print(silo_mode_delegation.get_current_mode())

Important Configuration Files

  • pyproject.toml: Python project configuration
  • setup.cfg: Python package metadata
  • .github/: CI/CD workflows
  • devservices/config.yml: Local service configuration
  • .pre-commit-config.yaml: Pre-commit hooks configuration
  • codecov.yml: Code coverage configuration

File Location Map

Backend

  • Models: src/sentry/models/{model}.py
  • API Endpoints: src/sentry/api/endpoints/{resource}.py
  • Serializers: src/sentry/api/serializers/models/{model}.py
  • Tasks: src/sentry/tasks/{category}.py
  • Integrations: src/sentry/integrations/{provider}/
  • Permissions: src/sentry/api/permissions.py
  • Feature Flags: src/sentry/features/permanent.py or temporary.py
  • Utils: src/sentry/utils/{category}.py

Tests

  • Python: tests/ mirrors src/ structure
  • Fixtures: fixtures/{type}/
  • Factories: tests/sentry/testutils/factories.py

Integration Development

Adding Integration

  1. Create dir: src/sentry/integrations/{name}/
  2. Required files:
    • __init__.py
    • integration.py (inherit from Integration)
    • client.py (API client)
    • webhooks/ (if needed)
  3. Register in src/sentry/integrations/registry.py
  4. Add feature flag in temporary.py

Integration Pattern

# src/sentry/integrations/example/integration.py
from sentry.integrations import Integration, IntegrationProvider

class ExampleIntegration(Integration):
    def get_client(self):
        from .client import ExampleClient
        return ExampleClient(self.metadata['access_token'])

class ExampleIntegrationProvider(IntegrationProvider):
    key = "example"
    name = "Example"
    features = ["issue-basic", "alert-rule"]

    def build_integration(self, state):
        # OAuth flow handling
        pass

Contributing Guidelines

  1. Follow existing code style
  2. Write comprehensive tests
  3. Update documentation
  4. Add feature flags for experimental features
  5. Consider backwards compatibility
  6. Performance test significant changes

Common Gotchas

  1. Hybrid Cloud: Check silo mode before cross-silo queries
  2. Feature Flags: Always add for new features
  3. Migrations: Test rollback, never drop columns immediately
  4. Celery: Always handle task failures/retries
  5. API: Serializers can be expensive, use @attach_scenarios
  6. Tests: Use self.create_* helpers, not direct model creation
  7. Permissions: Check both RBAC and scopes

Useful Resources

Notes for AI Assistants

  • This is a large, complex codebase with many interconnected systems
  • Always consider the impact of changes on performance and scalability
  • Many features are gated behind feature flags for gradual rollout
  • The codebase follows Django patterns but with significant customization
  • Database migrations require special care due to the scale of deployment
  • ALWAYS use pre-commit for linting instead of individual tools
  • Check silo mode before making cross-silo queries
  • Use decision trees above for common user requests
  • Follow the anti-patterns section to avoid common mistakes