> ## Documentation Index > Fetch the complete documentation index at: https://mintlify.com/bitwarden/server/llms.txt > Use this file to discover all available pages before exploring further. # Troubleshooting Guide > Common issues, diagnostic procedures, and solutions for Bitwarden Server This guide helps diagnose and resolve common issues with Bitwarden Server deployments. For each problem, we provide symptoms, root causes, and step-by-step solutions. ## Diagnostic Tools ### Service Health Checks ```bash theme={null} # Check container status docker ps -a | grep bitwarden # Test health endpoints curl http://localhost/healthz curl http://localhost/healthz/extended # View service logs docker logs --tail 100 bitwarden-api docker logs --tail 100 bitwarden-identity docker logs --tail 100 bitwarden-mssql ``` ### Database Connectivity ```bash theme={null} # Test SQL Server connection docker exec -it bitwarden-mssql /opt/mssql-tools18/bin/sqlcmd \ -S localhost -U sa -P "${SA_PASSWORD}" -C \ -Q "SELECT @@VERSION;" # Check database state docker exec -it bitwarden-mssql /opt/mssql-tools18/bin/sqlcmd \ -S localhost -U sa -P "${SA_PASSWORD}" -C \ -Q "SELECT name, state_desc FROM sys.databases WHERE name = 'vault';" ``` ### Network Diagnostics ```bash theme={null} # Test service connectivity from API container docker exec bitwarden-api curl -I http://identity:5000 # Check DNS resolution docker exec bitwarden-api nslookup identity # Verify port bindings netstat -tulpn | grep -E '(80|443|5000)' ``` ## Authentication Issues ### Login Failures **Symptoms:** * Users cannot log in * "Invalid username or password" errors * Timeout during authentication **Common Causes:** **Check:** Identity service status ```bash theme={null} docker ps | grep identity docker logs bitwarden-identity --tail 50 ``` **Solution:** ```bash theme={null} docker-compose restart identity ``` If service won't start, check configuration: ```bash theme={null} docker exec bitwarden-identity cat /app/appsettings.json ``` **Check:** Database connectivity from Identity service ```bash theme={null} docker exec bitwarden-identity cat /etc/bitwarden/config.yml ``` **Solution:** Verify connection string in `globalSettings.sqlServer.connectionString`: ```yaml theme={null} database: connectionString: "Server=mssql;Database=vault;User Id=sa;Password=***" ``` **Symptoms:** SSL/TLS errors, "Invalid token" messages **Check:** Certificate configuration ```bash theme={null} ls -la /etc/bitwarden/identity/ docker logs bitwarden-identity | grep -i certificate ``` **Solution:** Regenerate data protection certificates: ```bash theme={null} ./bitwarden.sh rebuild ``` **Symptoms:** Login works intermittently, 429 errors **Check:** Rate limit configuration in `appsettings.json` ```json theme={null} { "IpRateLimitOptions": { "GeneralRules": [ { "Endpoint": "post:/connect/token", "Period": "1m", "Limit": 10 } ] } } ``` **Solution:** Adjust rate limits or whitelist specific IPs: ```json theme={null} { "IpRateLimitOptions": { "IpWhitelist": ["10.0.0.0/8"] } } ``` ### Two-Factor Authentication Issues **Symptoms:** * 2FA codes not working * Email 2FA not received * Authenticator app code rejected **Diagnostic Steps:** Verify email configuration: ```bash theme={null} docker exec bitwarden-api cat /etc/bitwarden/config.yml | grep -A 10 mail ``` Test email service: ```bash theme={null} docker logs bitwarden-api | grep -i "mail\|smtp\|sendgrid" ``` TOTP codes require accurate time: ```bash theme={null} # Check server time docker exec bitwarden-identity date # Sync if needed sudo ntpdate -s time.nist.gov ``` 2FA endpoints have rate limiting: ```json theme={null} { "Endpoint": "post:/two-factor/send-email", "Period": "10m", "Limit": 5 } ``` ## Vault Access Issues ### Cannot Access Items **Symptoms:** * Vault appears empty * Specific items not visible * "Access denied" errors **Troubleshooting:** ```bash theme={null} # Check user permissions in database docker exec -it bitwarden-mssql /opt/mssql-tools18/bin/sqlcmd \ -S localhost -U sa -P "${SA_PASSWORD}" -C -Q " SELECT u.Email, ou.Status, ou.Type FROM [dbo].[User] u JOIN [dbo].[OrganizationUser] ou ON u.Id = ou.UserId WHERE u.Email = 'user@example.com'; " # Check collection access docker exec -it bitwarden-mssql /opt/mssql-tools18/bin/sqlcmd \ -S localhost -U sa -P "${SA_PASSWORD}" -C -Q " SELECT c.Name, cg.ReadOnly, cg.HidePasswords FROM [dbo].[Collection] c JOIN [dbo].[CollectionGroup] cg ON c.Id = cg.CollectionId WHERE c.OrganizationId = ''; " ``` ### Sync Failures **Symptoms:** * Changes not syncing across devices * "Sync failed" errors * Stale data displayed **Solutions:** ```bash theme={null} docker ps | grep notifications curl http://localhost:5000/healthz ``` Check for WebSocket errors in browser console Test WebSocket endpoint: ```bash theme={null} wscat -c ws://localhost/notifications/hub ``` For Azure Notification Hub deployments: ```json theme={null} { "notificationHub": { "connectionString": "Endpoint=sb://...", "hubName": "bitwarden" } } ``` ## Performance Issues ### Slow Response Times **Symptoms:** * API requests take >2 seconds * Timeout errors * High CPU/memory usage **Diagnostic Steps:** ```bash theme={null} # Check resource usage docker stats # Identify slow queries docker exec -it bitwarden-mssql /opt/mssql-tools18/bin/sqlcmd \ -S localhost -U sa -P "${SA_PASSWORD}" -C -Q " SELECT TOP 10 total_elapsed_time/execution_count AS avg_time_ms, execution_count, SUBSTRING(st.text, (qs.statement_start_offset/2)+1, ((CASE qs.statement_end_offset WHEN -1 THEN DATALENGTH(st.text) ELSE qs.statement_end_offset END - qs.statement_start_offset)/2) + 1) AS statement_text FROM sys.dm_exec_query_stats AS qs CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) AS st ORDER BY total_elapsed_time/execution_count DESC; " # Check for blocking docker exec -it bitwarden-mssql /opt/mssql-tools18/bin/sqlcmd \ -S localhost -U sa -P "${SA_PASSWORD}" -C -Q " SELECT * FROM sys.dm_exec_requests WHERE blocking_session_id <> 0; " ``` **Solutions:** **Index Fragmentation:** ```sql theme={null} -- Check fragmentation SELECT OBJECT_NAME(object_id), avg_fragmentation_in_percent FROM sys.dm_db_index_physical_stats(DB_ID(), NULL, NULL, NULL, 'LIMITED') WHERE avg_fragmentation_in_percent > 30; -- Rebuild indexes EXEC sp_MSforeachtable 'ALTER INDEX ALL ON ? REBUILD'; ``` **Statistics Update:** ```sql theme={null} EXEC sp_updatestats; ``` Increase connection pool size in connection string: ``` Server=mssql;Database=vault;User Id=sa;Password=***;Min Pool Size=10;Max Pool Size=100; ``` Increase container resources: ```yaml theme={null} services: api: deploy: resources: limits: cpus: '2.0' memory: 2G reservations: cpus: '1.0' memory: 1G ``` ### High Memory Usage **Check Memory Usage:** ```bash theme={null} # Container memory docker stats --no-stream # Host memory free -h ``` **Solutions:** * Reduce log verbosity (change to `Warning` level) * Implement log rotation with size limits * Clear old logs: `find /etc/bitwarden/logs -mtime +7 -delete` * Increase container memory limits ## Database Issues ### Database Connection Errors **Error Message:** `Login failed for user 'sa'` **Solutions:** Check SA password in environment: ```bash theme={null} docker exec bitwarden-mssql printenv | grep SA_PASSWORD ``` ```bash theme={null} docker exec -it bitwarden-mssql /opt/mssql-tools18/bin/sqlcmd \ -S localhost -U sa -P "${OLD_PASSWORD}" -C \ -Q "ALTER LOGIN sa WITH PASSWORD='${NEW_PASSWORD}';" ``` Update configuration with new password. ```bash theme={null} docker logs bitwarden-mssql --tail 50 ``` Look for: * "SQL Server is now ready for client connections" * Error messages about recovery or startup ### Migration Failures **Error:** `Server is in script upgrade mode` **Solution:** The migrator includes automatic retry logic (up to 10 attempts). If it persists: ```bash theme={null} # Check for blocking processes docker exec -it bitwarden-mssql /opt/mssql-tools18/bin/sqlcmd \ -S localhost -U sa -P "${SA_PASSWORD}" -C -Q " SELECT session_id, blocking_session_id, wait_type, wait_time FROM sys.dm_exec_requests WHERE blocking_session_id <> 0; " # Kill blocking session if found KILL ; ``` **Error:** Migration script timeout **Solution:** For large migrations, increase timeout: * Default: 5 minutes * Large migrations: 60 minutes (automatic for `noTransactionMigration`) See `util/Migrator/DbMigrator.cs:131` for configuration. ### Database Corruption **Symptoms:** * CHECKDB errors * Consistency errors in logs * Random query failures **Diagnostic:** ```sql theme={null} DBCC CHECKDB (vault) WITH NO_INFOMSGS; ``` **Recovery:** ```sql theme={null} DBCC CHECKDB (vault, REPAIR_REBUILD); ``` If automatic repair fails, restore from backup: ```bash theme={null} ./bitwarden.sh stop # Restore database (see Backup/Restore guide) ./bitwarden.sh start ``` ## Service-Specific Issues ### API Service Issues **Symptoms:** * 500 Internal Server Error * API endpoints not responding * Unhandled exceptions in logs **Implementation Reference:** `src/SharedWeb/Utilities/ExceptionHandlerFilterAttribute.cs` **Error Types:** | Exception Type | HTTP Status | Meaning | | -------------------------------- | ----------- | ------------------------ | | BadRequestException | 400 | Invalid request data | | GatewayException | 502 | Upstream service failure | | NotFoundException | 404 | Resource not found | | SecurityTokenValidationException | 401 | Invalid/expired token | | UnauthorizedAccessException | 401 | Insufficient permissions | | (Other) | 500 | Unhandled error | **Check Logs:** ```bash theme={null} docker logs bitwarden-api | grep -i "error\|exception" | tail -50 ``` ### Identity Service Issues **Symptoms:** * Token generation failures * OpenID configuration errors * Certificate errors **Check Configuration:** ```bash theme={null} # Verify Identity configuration docker exec bitwarden-identity cat /etc/bitwarden/config.yml # Test OpenID endpoint curl http://localhost/.well-known/openid-configuration ``` ### Admin Portal Issues **Symptoms:** * Cannot access admin portal * "Access denied" on admin pages * System information not loading **Solutions:** * Verify admin user has proper role * Check admin service logs * Ensure license is valid and not expired ## SSL/TLS Issues ### Certificate Errors **Symptoms:** * SSL handshake failures * Certificate validation errors * "NET::ERR\_CERT\_AUTHORITY\_INVALID" **Solutions:** ```bash theme={null} openssl s_client -connect yourdomain.com:443 -servername yourdomain.com ``` Ensure full certificate chain is installed: ```bash theme={null} cat server.crt intermediate.crt root.crt > fullchain.pem ``` ```bash theme={null} openssl x509 -in /etc/bitwarden/ssl/certificate.crt -noout -dates ``` ```bash theme={null} # Copy new certificate cp new-cert.crt /etc/bitwarden/ssl/certificate.crt cp new-key.key /etc/bitwarden/ssl/private.key # Restart nginx docker-compose restart nginx ``` ## File Storage Issues ### Attachment Upload Failures **Symptoms:** * "Failed to upload" errors * Attachments not appearing * Storage errors in logs **Check Configuration:** ```bash theme={null} # Verify storage configuration docker exec bitwarden-api cat /etc/bitwarden/config.yml | grep -A 5 attachment ``` **For Local Storage:** ```bash theme={null} # Check permissions ls -la /etc/bitwarden/core/attachments/ chown -R bitwarden:bitwarden /etc/bitwarden/core/attachments/ # Check disk space df -h /etc/bitwarden/ ``` **For Azure Blob Storage:** ```bash theme={null} # Test connection string az storage blob list \ --account-name \ --container-name attachments \ --connection-string "${CONNECTION_STRING}" ``` ## Logging and Debugging ### Enable Debug Logging Temporarily enable verbose logging: ```json theme={null} { "Logging": { "LogLevel": { "Default": "Debug", "Bit.Api": "Trace", "Bit.Core": "Debug" } } } ``` Debug/Trace logging generates significant log volume. Only enable for troubleshooting and revert to `Warning` or `Information` afterwards. ### Collect Diagnostic Information ```bash theme={null} #!/bin/bash # Diagnostic collection script DIAG_DIR="bitwarden-diagnostics-$(date +%Y%m%d_%H%M%S)" mkdir -p "$DIAG_DIR" # Container status docker ps -a > "$DIAG_DIR/containers.txt" # Logs from all services for service in api identity admin notifications events; do docker logs "bitwarden-$service" > "$DIAG_DIR/$service.log" 2>&1 done # System info uname -a > "$DIAG_DIR/system.txt" df -h >> "$DIAG_DIR/system.txt" free -h >> "$DIAG_DIR/system.txt" # Network info docker network inspect bitwarden_default > "$DIAG_DIR/network.json" # Create archive tar -czf "$DIAG_DIR.tar.gz" "$DIAG_DIR" rm -rf "$DIAG_DIR" echo "Diagnostics collected: $DIAG_DIR.tar.gz" ``` ## Getting Help ### Before Requesting Support Review service logs for error messages and stack traces. Search this documentation and Bitwarden community forums. Double-check configuration files against documentation. Try to reproduce the issue with minimal configuration. ### Information to Provide When requesting support, include: * Bitwarden Server version * Deployment type (Docker, Kubernetes, etc.) * Operating system and version * Relevant error messages and logs * Steps to reproduce the issue * Recent changes to configuration ## Related Resources * [Health Check Configuration](/operations/health-checks) * [Logging Configuration](/operations/logging) * [Backup and Restore](/operations/backup-restore) * [Update Procedures](/operations/updates)