Which AWS Checks Survive Local Emulation: AWS Emulator, LocalStack, and Moto Under One Test Suite

Which AWS Checks Survive Local Emulation: AWS Emulator, LocalStack, and Moto Under One Test Suite

pr0h0••22 min read•tech
AI Usage (87%)
On this page

Introduction: Which AWS Checks Survive Local Emulation

A report published by 디지털투데이 on 11 November 2026 says AWS released an open-source "AWS Emulator" for validating code locally instead of against live AWS services. That is the only public claim I can verify from the source. This post answers the practical question that follows: which AWS checks actually survive local emulation when Moto, LocalStack, and the reported AWS Emulator are run against the same test corpus, and where service-fidelity gaps make a green local run unsafe to treat as deploy approval. My position up front:

Emulators validate contracts, not behavior. A green local run is evidence that request shapes, ARNs, and state machinery are wired correctly. It says nothing about whether the deployment works, and it never proves the permissions model holds.

To make that concrete, I built a test corpus that runs unchanged against three backends — Moto, LocalStack, and a real AWS account as the control — and recorded where results diverge. The reported AWS Emulator is a fourth column marked "unknown" because the report does not name the repository, supported services, license, or enforcement model. I would rather not invent those details. The harness is written so the emulator can be graded against both baselines the day its interface is public.

What the Report Says, and What It Does Not

The confirmed claim: an open-source AWS Emulator for local code validation

The report says AWS released an open-source emulator for local code validation, aimed at workflows that would otherwise need live AWS services. That claim is load-bearing and narrow. Everything below about the category — local AWS emulation — is based on tools I can install and read source for. Everything about the new product is explicitly flagged.

The unconfirmed parts: service coverage, IAM fidelity, release channel, license

Here is what the report does not tell me, and what each gap would change:

UnknownWhy it decides your workflow
Service coverage listS3/SQS/DynamoDB only is a contract test harness. Add Step Functions, EventBridge Scheduler, and ECS and it is a different tier of tool.
IAM enforcement modelWithout request-time policy evaluation, every authorization test is theater. See Gap 1.
Transport (in-process patch vs. listening endpoint)Determines whether non-Python code, presigned URLs, and curl-style checks can be tested at all.
License and release channelAn aws-samples reference project, a GA service, and an experimental repo have three different support postures.
Versioning and fidelity commitmentsAn emulator without a documented fidelity matrix is a moving target you cannot regression-test against.

Why the gap between those two lists decides how you can use it

The difference matters because the cost of being wrong is asymmetric. If a contract test passes locally but fails on deploy, you lose ten minutes. If an IAM test passes locally because the emulator never evaluates policies, you may ship a privilege escalation path with a green pipeline. The second failure mode is the one that gets people fired, and it is the one these tools are most often mis-sold as preventing.

Three Tools, Three Fidelity Models

Moto: in-process patching of botocore, not a running service

Moto intercepts botocore at the transport layer inside your Python process. mock_aws() swaps the HTTP sender, so boto3.client("s3").list_buckets() never opens a socket. It runs in milliseconds, needs no daemon, and isolates cleanly per test. It is also Python-only, which quietly rules out testing your Go worker, your Node Lambda bundle, or the curl command in your runbook.

Moto ships moto_server, which turns it into an HTTP endpoint. Most teams never use it. That is a mistake worth understanding: once Moto runs in server mode, a large class of previously impossible checks becomes possible.

LocalStack: a real HTTP endpoint backed by service emulators

LocalStack listens on http://localhost:4566 and speaks HTTP. Point your SDK at it with an endpoint URL, and the code path exercised is the real one: no DNS, but real TCP with signing, retries, and connection handling. It works with any language, which is why it tends to win in polyglot repos. You pay for it in startup latency (a container, plus Docker socket access for Lambda) and in state that persists across tests unless you do something about it.

AWS Emulator: architecture unknown, so grade it against both baselines

Since I cannot verify the emulator's transport, I do not put it in either camp. The harness treats the two existing tools as calibration points. If the reported emulator listens on a port, expect it to reproduce the LocalStack column. If it patches the SDK, expect the Moto column. If it does something else — a proxy or a VPC layer — both columns still give you bounds.

Why the transport layer determines which checks can survive

This is the single most underrated design question in the comparison. Consider a presigned S3 URL:

url = s3.generate_presigned_url(
    "get_object", Params={"Bucket": "b", "Key": "k"}, ExpiresIn=60
)
requests.get(url)   # works only if something is listening

Under in-process Moto, generate_presigned_url succeeds and returns something plausible. The requests.get then fails because nothing is bound to a port. Under LocalStack the same request returns the object. Same test, same code, opposite result — and the difference is not service fidelity, it is transport. Any check that crosses a process boundary (a curl health check, a presigned URL, a webhook receiver, a language runtime you do not own) is a transport test wearing a fidelity costume.

The Thesis: Emulators Validate Contracts, Not Behavior

What a passing local test actually proves

A green run against an emulator proves a real and non-trivial set of things:

  • your request parameters serialize and validate against the SDK's model
  • your response parsing handles the documented shape, including nested error structures
  • your resource naming, ARN construction, and queue-URL handling are internally consistent
  • your state machine flows (create → configure → use → delete) do not deadlock on your own ordering
  • your configuration plumbing resolves region, credentials, and endpoint overrides correctly

That is a lot. Most production incidents I have seen in AWS glue code live here: wrong parameter name, wrong region, an ARN assembled by string concatenation, a queue URL that needed to be stored rather than rebuilt.

The four things local emulation can never prove on its own

  1. Authorization. No community-tier emulator evaluates IAM policies at request time. Creating a policy and then succeeding at a call proves the policy document parsed, nothing more.
  2. Quotas and throttling. Emulators do not enforce per-account service quotas or throughput limits, so ProvisionedThroughputExceededException, ThrottlingException, and backoff paths stay untested.
  3. Eventual consistency and cross-service timing. DynamoDB's read-after-write model, S3 list consistency, SQS at-least-once redelivery under visibility timeouts — all simplified or absent.
  4. Managed-service internals. Object Lock compliance mode, globally unique S3 bucket names, VPC endpoint policies, TLS enforcement, CloudTrail audit records, cross-account trust evaluation.

You can write tests for all four. You just cannot run them locally and call them evidence.

Building One Test Suite That Targets All Three Backends

Choosing the seam: endpoint URL versus mock layer versus adapter

Three options exist, and only one scales across backends:

  • Import Moto in the test body. Fast, permanently Moto-only. Avoid.
  • Endpoint URL everywhere. Requires Moto in server mode to be comparable, and adds a container to every developer's machine.
  • A client_factory fixture selected by environment variable. The test body never imports either library. This is the one.

Since botocore 1.29, boto3 honours AWS_ENDPOINT_URL and AWS_ENDPOINT_URL_<SERVICE> (SDK reference), which means for HTTP backends you often need no code change at all — just environment.

A backend-agnostic fixture design

## tests/emu/conftest.py




BACKEND = os.environ.get("EMU_BACKEND", "moto")
ENDPOINT = os.environ.get("EMU_ENDPOINT", "http://localhost:4566")
REGION = os.environ.get("AWS_REGION", "us-east-1")

@pytest.fixture
def client_factory():
    if BACKEND == "moto":
        from moto import mock_aws
        with mock_aws():                      # function scope: state resets per test
            yield _factory(None)
    elif BACKEND == "emulator":               # http endpoint, any implementation
        yield _factory(ENDPOINT)
    elif BACKEND == "aws":                    # control lane, real account
        yield _factory(None)
    else:
        raise SystemExit(f"unknown EMU_BACKEND={BACKEND}")

def _factory(endpoint):
    def make(service, **kw):
        if endpoint:
            kw.setdefault("endpoint_url", endpoint)
            kw.setdefault("aws_access_key_id", "test")
            kw.setdefault("aws_secret_access_key", "test")
        return boto3.client(service, region_name=REGION, **kw)
    return make

Two design decisions worth defending. First, Moto is function-scoped: cheap (~ms) and perfectly isolated. LocalStack is not reset per test; restarting a container per test would dominate CI. Instead, name every resource with a per-test prefix and clean up in a fixture finaliser. Second, the same fixture serves the real-AWS control lane with endpoint=None, so the control is not a special case in assertions — only in wiring.

Harness code for Moto (mock_aws) and HTTP-endpoint backends

Checks return data, not booleans. That trick makes cross-backend diffs readable:

## tests/emu/checks/tier1_sqs.py
def check_sqs_roundtrip(make):
    sqs = make("sqs")
    url = sqs.create_queue(QueueName="contract-q")["QueueUrl"]
    sqs.send_message(
        QueueUrl=url,
        MessageBody="hello",
        MessageAttributes={"trace": {"DataType": "String", "StringValue": "abc123"}},
    )
    msg = sqs.receive_message(
        QueueUrl=url, MessageAttributeNames=["All"], MaxNumberOfMessages=1
    )["Messages"][0]
    return {
        "id": "tier1.sqs.roundtrip",
        "body": msg["Body"],
        "trace_attr": msg.get("MessageAttributes", {})
                          .get("trace", {}).get("StringValue"),
        "url_shape": url.replace("contract-q", "<name>"),
    }

A collector turns those dicts into a JSON report per backend:

## tests/emu/conftest.py (continued)


@pytest.fixture(scope="session")
def emu_report(request):
    records = {}
    yield records
    path = pathlib.Path(request.config.getoption("--emu-report"))
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_text(json.dumps({"backend": BACKEND, "checks": records}, indent=2))

The comparison script diffs two reports key by key and prints only divergences. You are not chasing green; you are cataloguing divergence.

Pinning one test corpus to every backend so results are comparable

The corpus must be written once, with no backend conditionals inside it. If a check needs if BACKEND == "moto", that is a finding, not a workaround — record the deviation in the expected-divergence file instead. Practical rules:

  • pin exact versions in your lockfile (moto, boto3, the LocalStack image tag); emulator behavior shifts between minors
  • keep the corpus small and brutal: 25–40 checks, one per behavior you actually depend on
  • never assert on an emulator-generated identifier; normalise it (as url_shape does above)
  • store the raw report as a CI artifact so a divergence is a diff, not an argument

Recording the diff instead of asserting a single green run

A useful output looks like this (shape shown; exact values track your pins):

backend=moto
  tier1.sqs.roundtrip        ok     url_shape=https://sqs.us-east-1.amazonaws.com/123456789012/<name>
  tier2.s3.presigned_get     DIFF   ConnectionRefused
  tier3.iam.deny_unpermitted DIFF   expected AccessDenied, got success

backend=localstack
  tier1.sqs.roundtrip        ok     url_shape=http://localhost:4566/000000000000/<name>
  tier2.s3.presigned_get     ok
  tier3.iam.deny_unpermitted DIFF   expected AccessDenied, got success

backend=aws
  tier1.sqs.roundtrip        ok     url_shape=https://sqs.us-east-1.amazonaws.com/123456789012/<name>
  tier2.s3.presigned_get     ok
  tier3.iam.deny_unpermitted ok

Note the two url_shape values: Moto mints 123456789012, LocalStack mints 000000000000. Both are correct for their world and wrong for production. Any test that hard-codes an account ID is asserting an emulator fact.

The Check Taxonomy

Tier 1 — request and response shape contracts

Serialization, required parameters, response keys your code reads, error Code values your code branches on. These should run on every commit, because they fail for reasons code review misses.

def check_s3_put_returns_etag(make):
    s3 = make("s3")
    s3.create_bucket(Bucket="contract-bucket")
    r = s3.put_object(Bucket="contract-bucket", Key="k", Body=b"x")
    return {"id": "tier1.s3.etag", "etag_present": bool(r.get("ETag"))}

Tier 2 — stateful CRUD and resource lifecycle semantics

Create → update → read → delete, with the ordering your application actually uses. This is where emulators earn their keep: it is cheap to prove your Terraform-adjacent Python does not delete a queue before draining it.

Tier 3 — IAM policy evaluation and authorization denial

The check is simple and the result is damning. Create an IAM user with no permissions, mint an access key, sign a client with it, and attempt a privileged call:

def check_iam_denies_unpermitted(make, make_as):
    iam = make("iam")
    iam.create_user(UserName="nobody")
    keys = iam.create_access_key(UserName="nobody")["AccessKey"]
    as_nobody = make_as(keys["AccessKeyId"], keys["SecretAccessKey"])
    try:
        as_nobody("s3").create_bucket(Bucket="should-not-exist")
        return {"id": "tier3.iam.deny", "decision": "ALLOWED"}
    except Exception as e:
        return {"id": "tier3.iam.deny", "decision": type(e).__name__,
                "code": getattr(e, "response", {}).get("Error", {}).get("Code")}

If that returns ALLOWED on your emulator, tier 3 is not testable there. Full stop. LocalStack's request-time IAM enforcement is a Pro/Team feature (ENFORCE_IAM), not a community one, and Moto does not evaluate policies at all — it stores them. That is not a criticism; it is a scope boundary you have to encode in your pipeline.

Tier 4 — failure modes, throttling, retries, eventual consistency

Try to provoke ProvisionedThroughputExceededException on a DynamoDB table with 1 WCU, or a ThrottlingException from SQS. Locally you will get clean successes because emulators do not model capacity. Your exponential backoff code is therefore untested by local runs, and the first time it executes will be in production.

Tier 5 — service-specific behavior (S3, SQS, DynamoDB, Lambda, EventBridge)

Bucket-name global uniqueness across all AWS accounts, Object Lock compliance mode, DynamoDB conditional writes under contention, Lambda cold-start concurrency, EventBridge scheduled-invocation timing and at-least-once delivery. Some are modelled; most are approximated. The safe move is to probe each one you depend on and record the result rather than assume.

Results: Which Checks Survive Local Emulation

Results table: tier by backend, pass / partial / fail

Legend: P pass, p partial (shape correct, semantics simplified), F fail. ᵗ = came out of running the harness; ᵈ = derived from documented scope of the tool, worth re-verifying on your pins.

TierCheck familyMotoLocalStack (community)LocalStack (ENFORCE_IAM, Team)AWS Emulator
1Request/response shapeP ᵗP ᵗPunknown
2Stateful CRUD, lifecycleP ᵗP ᵗPunknown
2Presigned URL round-tripF ᵗP ᵗPunknown — transport dependent
3IAM denialF ᵈF ᵗPunknown — decisive question
4Throttling / backoffF ᵈF ᵈF ᵈunknown
4Eventual consistencyF ᵈF ᵈF ᵈunknown
5S3 global name uniquenessF ᵈF ᵈF ᵈunknown
5Lambda direct invokep ᵗP ᵗPunknown
5SQS→Lambda event source mappingp ᵗp ᵗpunknown
5EventBridge scheduled timingF ᵈp ᵈp ᵈunknown

The reported AWS Emulator column is honest: I have not published a number for a tool whose interface I cannot verify. The point of the harness is that this column fills itself in within an afternoon once the repository is readable.

Checks that survive everywhere

Tier 1 and most of tier 2. Parameter serialization, response key access, ARN construction, CRUD ordering, tag handling, idempotency at the API level. These cover most of your glue-code risk and they are genuinely tested.

Checks that survive only on a running-endpoint emulator

Anything crossing a process boundary: presigned URLs, curl in a smoke script, a Go or Node service in the same repo, a webhook listener, or a Lambda handler invoked through a real runtime. This is the strongest practical argument for LocalStack over Moto in polyglot repos — and the reason to care whether the reported AWS Emulator listens on a port.

Checks that survive nowhere locally

IAM denial (community tier), quotas and throttling, read-after-write timing, cross-account trust, SCP boundaries, global namespace collisions, TLS and VPC endpoint policies, CloudTrail audit emission. These must run against a real account, in a sandbox, with cleanup.

Reproducing the matrix yourself in a single command

pip install "moto[all]" boto3 pytest requests
docker run -d --name ls -p 4566:4566 \
  -e SERVICES=s3,sqs,sns,dynamodb,iam,sts,events \
  localstack/localstack:3

EMU_BACKEND=moto       pytest tests/emu -q --emu-report out/moto.json
EMU_BACKEND=emulator   pytest tests/emu -q --emu-report out/localstack.json
EMU_BACKEND=aws        pytest tests/emu -q --emu-report out/aws.json \
                       --aws-profile hackyjs-sandbox

python tools/diff_matrix.py out/*.json --baseline out/aws.json

Add --endpoint-url http://localhost:4566 when you want to run the same corpus through the AWS CLI, which is a useful cross-check that your Python code is not the thing under test.

The Fidelity Gaps That Actually Bite

Gap 1 — IAM is the largest divergence and the most dangerous one

No emulator in the free tier evaluates identity policies at request time. LocalStack gates enforcement behind Pro/Team; Moto stores policies without consulting them. This gap produces security incidents with green pipelines, because a test named test_reader_cannot_write that passes on an emulator is worse than no test: it manufactures confidence.

⚠️

Never let a test named after an authorization outcome run only against an emulator without an explicit marker in its name or the expected-divergence file. A false green here is a security finding, not a flake.

Gap 2 — read-after-write and eventual consistency

DynamoDB's model (strongly consistent reads on the item, eventually consistent on GSIs), S3 list consistency, SQS visibility-timeout redelivery — all simplified. Code that "just works" locally because reads are immediately visible can fail intermittently under real load. The failure is not a crash; it is a race that appears once a week in production and never in CI.

Gap 3 — pagination, limits, and quota behavior

Emulators rarely enforce per-account quotas (bucket counts, queue counts, IAM entity limits) or the awkward maximums. A MaxKeys=1000 loop that never terminates because the emulator returns fewer pages, or a request that exceeds a real 256 KB payload limit and is silently accepted locally, are both classic. Probe limits explicitly by sending the boundary value and comparing error codes.

Gap 4 — error shapes, retry semantics, and throttling

Error Code values mostly match; error conditions do not. Your except client.exceptions.ThrottlingException branch has, in all likelihood, never executed locally. If a retry path matters, test it by injecting the failure at your own boundary — a wrapper that raises the modelled exception — and make the test name say the injection is simulated, not AWS-observed.

Gap 5 — container and managed-integration services

Lambda under LocalStack needs the Docker socket; event source mappings and container-image functions are the shakiest area. ECS, RDS, and other managed services are either absent or so approximated that testing against them tells you about the emulator, not AWS. My rule: if a service requires the emulator to simulate a hypervisor or a scheduler, treat the local result as a contract check on your request payload and nothing more.

A Worked Example: SQS, Lambda, and IAM in Three Backends

The stack under test (queue, consumer, least-privilege role)

A queue with a dead-letter queue and a redrive policy, a Python Lambda consuming it, and an execution role granting exactly sqs:ReceiveMessage, sqs:DeleteMessage, sqs:GetQueueAttributes on that queue ARN. This is the shape of about half the event-driven stacks I audit, and it exercises every tier at once.

def check_stack_wiring(make):
    sqs, lam, iam = make("sqs"), make("lambda"), make("iam")
    dlq = sqs.create_queue(QueueName="orders-dlq")["QueueUrl"]
    dlq_arn = sqs.get_queue_attributes(
        QueueUrl=dlq, AttributeNames=["QueueArn"])["Attributes"]["QueueArn"]
    q = sqs.create_queue(QueueName="orders", Attributes={
        "RedrivePolicy": json.dumps({"deadLetterTargetArn": dlq_arn,
                                     "maxReceiveCount": "3"})})["QueueUrl"]
    q_arn = sqs.get_queue_attributes(
        QueueUrl=q, AttributeNames=["QueueArn"])["Attributes"]["QueueArn"]
    role = iam.create_role(RoleName="orders-consumer",
        AssumeRolePolicyDocument=json.dumps(TRUST))["Role"]["Arn"]
    return {"id": "tier2.stack.wiring", "queue_arn": q_arn, "role_arn": role}

Running the suite against Moto

EMU_BACKEND=moto pytest tests/emu -q. Fast, isolated, and the ARNs look like production. The redrive policy round-trips as an attribute string. Direct lambda.invoke works for a zipped handler if the runtime is available locally. But presigned URL checks fail, and any attempt to assert that the role cannot read another queue will fail in the dangerous direction — it can.

Running the suite against LocalStack

EMU_BACKEND=emulator pytest tests/emu -q against :4566. The wiring checks pass, ARNs carry 000000000000, presigned URLs work because something is listening. Watch the Lambda path: if create_event_source_mapping returns successfully but nothing is ever invoked, do not treat that as a production bug — record it as an emulator gap and invoke the handler directly to test the payload contract. That distinction is what the expected-divergence file is for.

Running the same suite against a real account as the control

EMU_BACKEND=aws pytest tests/emu -q with a sandbox profile, a prefix like emu-<runid>-, and mandatory teardown. This lane produces the baseline that gives the other two columns meaning. Without it, "passing" is undefined.

Reading the diffs and deciding what each result is worth

Three readings, three conclusions:

  • A check that passes on all three backends is a genuine contract guarantee. Promote it to a blocking gate.
  • A check that passes locally and fails against AWS is a fidelity gap. Record it with an owner and a review date — do not delete the test.
  • A check that fails locally and passes against AWS is the most interesting case: it usually means your local assumption about namespacing or timing is wrong, and it is a bug in your test, not the emulator.

Making Local Emulation Safe for Pre-Deploy Validation

Split contract tests from behavior tests in the pipeline

Physically separate them. tests/emu/ runs on every commit against the emulator with a hard time budget. tests/aws/ runs the tier-3/4/5 subset against a sandbox account on merge or nightly. The naming should make it impossible to confuse "validated locally" with "validated".

Maintain an explicit expected-divergence file

A checked-in YAML file mapping check id → backend → expected outcome and reason:

tier3.iam.deny_unpermitted:
  moto: ALLOWED
  emulator: ALLOWED
  reason: "no request-time policy evaluation at community tier"
  reviewed: 2026-11-11
  owner: platform-team
tier2.s3.presigned_get:
  moto: ConnectionRefused
  reason: "in-process transport has no listening socket"

CI fails when reality differs from this file. That gives you drift detection in both directions — including the day an emulator starts enforcing IAM, which is good news you want to notice rather than discover by accident.

Add a low-cost real-AWS smoke lane as the ground truth

Twenty-five checks, sandbox account, aggressive cleanup, run on merge. It costs almost nothing in dollars and it is the only lane that can prove denial works. Treat its failures as blocking for anything that touches authorization.

Cost and latency math for the CI split

Take a 400-test suite averaging 6 AWS calls per test — 2,400 calls per run.

LanePer-call costWall clock (serial)Money per run
Moto (in-process, ~2 ms/call)~0~5 s$0
LocalStack (:4566, ~5 ms/call)container startup ~10 s~22 s$0
Real AWS (40 ms RTT, 8-way parallel)~40 ms~96 s serial / ~12 s parallelwell under $0.01 at SQS request pricing

The headline: the cost argument for emulation is mostly latency and flakiness, not dollars, once you are inside the free tier. The real saving is not the AWS bill; it is the developer loop that runs in 5 seconds instead of 100, and the removal of credential plumbing as a CI failure mode.

A CI decision table: what blocks merge versus what warns

TierMoto / emulatorReal AWS laneMerge behaviour
1requiredoptionallocal failure blocks merge
2requiredoptionallocal failure blocks merge
3never blocksrequiredreal-lane failure blocks merge
4never blocksrequiredwarn, escalate if repeated
5best-effortrequired for covered serviceswarn
💪

If your test suite cannot run at all against a real account, that is the finding. Emulation is a fast inner loop, not a substitute for a lane that can actually deny you.

Anti-Patterns That Make Emulators Lie to You

Mocking the client library instead of the service

Stubbing boto3 with botocore.stub.Stubber asserts that your code calls what you told it to call. It never touches serialization or the service's validation. You are testing your own assumptions twice. Use an emulator, which at least parses the request.

Asserting on emulator-specific error strings and identifiers

assert arn.endswith(":000000000000:queue/orders") locks your tests to LocalStack. Normalise identifiers through a helper; assert on shape and suffix, not on the account namespace.

Treating emulator IAM as a security test

Covered above, repeated because it is the expensive mistake. If the emulator returns ALLOWED for a denied principal and your test passes because it expected success, you have written a test that documents a vulnerability as intended behavior.

Reading a green local run as deploy approval

A local run is a lint pass with better parsing. Deploy approval needs the real lane. Write that sentence into the pipeline docs before someone with a deadline invents a different reading.

What Would Change This Assessment

Signals to watch in the AWS Emulator repository and docs

  • a published service coverage matrix with an explicit fidelity column, not just "supported"
  • a limitations document that names IAM, throttling, and consistency as out of scope — negative documentation is the strongest signal of engineering honesty here
  • a conformance corpus run against real AWS in the project's own CI, which would let you inherit their divergence data instead of generating your own
  • a transport decision stated plainly: port listener or SDK patch
  • versioned releases tied to SDK model versions, so your pins mean something

What happens to the calculus if real IAM enforcement lands

If request-time policy evaluation arrives in the free tier, tier 3 partially moves local and the economics change substantially: authorization regressions get caught in the 5-second loop instead of the nightly lane. It would not eliminate the real-AWS lane — cross-account trust, SCPs, permission boundaries, and resource policies with conditions still need a real evaluator — but it would remove the most dangerous false green.

What stays true even then

Eventual consistency, quotas, global namespace, managed-service internals, cross-account and organizational boundaries, TLS and endpoint policies, audit records, and anything involving a scheduler or hypervisor. No local tool will model those, because modelling them means rebuilding AWS.

Conclusion

The reported release of an open-source AWS Emulator is good news for anyone who has waited ninety seconds for a CI job to fail on a malformed RedrivePolicy. Local emulation is the right place to catch contract bugs, and the existing options — Moto's in-process speed, LocalStack's real endpoint — are both worth having.

But the report tells us almost nothing about what the new tool actually enforces, and the difference between "covers S3 and SQS" and "evaluates IAM at request time" is the difference between a fast linter and a security control. Do not adopt it on the announcement. Adopt it the way you would adopt any emulator: build one corpus, run it against Moto, against LocalStack, and against a real sandbox account, and let the diff tell you which column it belongs in.

My position, stated plainly: run contract tests locally on every commit, run authorization tests against a real account on every merge, and keep a file that names the gap between them out loud. Emulators are excellent at telling you your code is well-formed. They will never tell you it is safe.

Further Reading

Share this post

More posts

Loading comments…