cosec-troubleshoot

name	cosec-troubleshoot
description	Diagnose CoSec authentication and authorization failures, including unexpected 401/403/429 responses, policy loading, JWT rejection, matcher mismatches, Redis limiters, and unexplained access decisions. Do not change behavior unless the user asks for a fix.

CoSec Troubleshooting

Find the first incorrect transition from request parsing to authentication, policy loading, matching, and response mapping. Gather evidence before proposing a policy or code change.

Start with observable facts

Capture the request method/path, response status and CoSec JSON reason, principal ID/authenticated state, app/space/tenant IDs, effective configuration, loaded policy IDs, and relevant startup/request logs. Redact tokens and secrets.

Enable focused logging:

logging:
  level:
    me.ahoo.cosec.authorization.SimpleAuthorization: DEBUG
    me.ahoo.cosec.audit: DEBUG

SimpleAuthorization identifies the matched policy/statement or role/permission. When AuditingAuthorization is active, the default sink emits structured deny events at WARN and allow events at DEBUG, including principal, request, decision, and matched rule when available. A custom Authorization bean is not wrapped automatically.

Interpret the status first

HTTP status	CoSec meaning
400	Invalid normalized request path
401	Authorization denied while the parsed principal is anonymous; invalid/expired tokens also fall back to anonymous and preserve the token reason
403	An authenticated principal was denied, or a regex matcher timed out and failed closed
429	A local or Redis rate limiter threw TooManyRequestsException
500	Unexpected authorization error; inspect the server exception

The authorization result alone does not choose 401 versus 403; the filter maps a denied anonymous principal to 401 and a denied authenticated principal to 403.

Decision order

SimpleAuthorization evaluates:

1. Root principal ID (cosec by default, overridden by JVM system property cosec.root) → allow.
2. Blacklist → explicit deny.
3. Global policies.
4. Principal-attached policies.
5. Role permissions for request.appId + request.spaceId.
6. No match → implicit deny.

Each policy tier evaluates matched policy-level conditions once, pools all statements, checks every deny before any allow, and stops at the first tier with a result. A deny only overrides allows in the same tier; a global allow prevents later principal or role denies from running.

Symptom guide

Unexpected 401

• Check whether the request actually carried a bearer token and whether parsing produced an authenticated principal.
• Match cosec.jwt.algorithm and secret to the issuer; only HMAC256/384/512 are supported.
• Check expiration and the response reason (Token Expired versus Token Invalid).
• If revocation is enabled, verify the cache capability supplied a real Redis-backed TokenStore; otherwise the starter warns that logout is ineffective.
• If the token is valid but the request remains anonymous, inspect the active SecurityContextParser and transport filter.

Unexpected 403 or missing allow

• Read the debug/audit match. An explicit deny is a matched deny; an implicit deny means no rule in any tier matched.
• Confirm a global policy loaded and its policy-level condition matched.
• Check the action's HTTP method, path, trailing slash, and captured variables.
• Check condition parts and the actual principal roles/attributes/tenant.
• Remember that inTenant.value is default, user, or platform, not a tenant ID.

Request allowed unexpectedly

• Check root bypass first.
• Find the first matched tier. A broad global allow short-circuits stricter principal and role rules.
• Verify the deny action and condition both match; array order cannot make an allow beat a same-tier deny.
• Confirm the intended policy was loaded rather than skipped or shadowed by a duplicate ID.

Policy file has no effect

• Confirm cosec.authorization.local-policy.enabled=true and the file matches local-policy.locations.
• Inspect startup errors: LocalPolicyLoader logs and skips malformed JSON, missing required fields, unknown matcher types, and constructor-time configuration errors rather than failing the application.
• Deferred matcher errors still load. For example, an unsupported part throws during authorization and the transport returns 500; reproduce it with DefaultPolicyEvaluator or the failing request.
• Local policies are deduplicated by ID after location expansion; assert that the expected ID and content loaded.
• LocalPolicyLoader does not serve authorization decisions directly. Enable init-repository or prepopulate PolicyRepository; use force-refresh only when overwriting repository state is intended.

Matcher mismatch

• Path variables use {id} and conditions read request.path.var.id; :id and request.pathVariable.id are invalid.
• Path SpEL templates use #{principal.id}. SpEL condition expressions use root properties such as context.principal.id; they are different evaluation roots.
• Valid part prefixes are singular request.header., request.attributes., request.path.var., and context.principal.attributes.. Unsupported parts throw IllegalArgumentException.
• Regex timeouts fail closed as 403. Simplify or bound the pattern instead of increasing exposure to ReDoS.

Rate limiting differs across instances

• rateLimiter and groupedRateLimiter are in-memory and per JVM.
• redisRateLimiter and redisGroupedRateLimiter are cluster-wide and require cache support plus StringRedisTemplate.
• Redis limiter outages fail open by default. Set the matcher's strictFailure: true only when availability loss should return 429.
• Check cosec.limiter.key-prefix, windowSeconds, grouping part, and whether multiple policies intentionally share an identical quota configuration.

Reproduce narrowly

Prefer the smallest existing test layer that crosses the failing boundary:

• matcher behavior: create the real factory with mapOf(...).asConfiguration() and test match;
• policy parsing: load the exact resource with LocalPolicyLoader, assert its ID, then run DefaultPolicyEvaluator.evaluate;
• decision order: test SimpleAuthorization with the smallest repositories needed;
• transport mapping: use the WebFlux, WebMVC, or Gateway filter tests for 401/403/429 behavior;
• Redis behavior: run the cache integration test with Redis.

In this repository, run a focused class first, for example:

./gradlew :cosec-core:test --tests "me.ahoo.cosec.policy.DefaultPolicyEvaluatorTest"

Use $cosec-policy-author to rewrite a confirmed-bad policy and $cosec-integration to change confirmed-bad Spring configuration. For diagnosis-only requests, report the root cause and evidence without modifying files.