Add a List Endpoint to Show Multiple Uploaded Files

Learning Outcomes

By the end of this lesson, you will be able to:

1. Build an API endpoint that lists multiple metadata records from DynamoDB.
2. Explain why Query is a better fit than Scan for this table design.
3. Filter results by bucket and optionally by object-key prefix.
4. Return a JSON array from Lambda through API Gateway.
5. Explain what LastEvaluatedKey means in plain terms.

---

The Core Idea

Your table is designed with:

• Partition key: bucket
• Sort key: object_key

That means the natural way to list items is to query by bucket, and optionally narrow the results with a sort-key prefix such as incoming/. DynamoDB Query requires the partition key to be matched, and it can optionally use a condition on the sort key, including begins_with. That makes it a much better fit than scanning the whole table.

---

Why Query instead of Scan

Use this rule:

• Query when you know the key pattern you want.
• Scan when you are reading the whole table and filtering afterward.

For this lesson, you already know the bucket, which is the partition key. A Scan reads the entire table and is less efficient for this pattern.

---

Part 1: Create the List Lambda Function

Create a new Lambda function and keep the same environment variable: TABLE_NAME=upload_metadata.

Use this Python code:

import json
import os
import decimal
import boto3
from boto3.dynamodb.conditions import Key

dynamodb = boto3.resource("dynamodb")
table = dynamodb.Table(os.environ["TABLE_NAME"])

def to_json_safe(value):
    if isinstance(value, list):
        return [to_json_safe(v) for v in value]
    if isinstance(value, dict):
        return {k: to_json_safe(v) for k, v in value.items()}
    if isinstance(value, decimal.Decimal):
        if value % 1 == 0:
            return int(value)
        return float(value)
    return value

def lambda_handler(event, context):
    query = event.get("queryStringParameters") or {}

    bucket = query.get("bucket")
    prefix = query.get("prefix")
    limit_raw = query.get("limit")

    if not bucket:
        return {
            "statusCode": 400,
            "headers": {"Content-Type": "application/json"},
            "body": json.dumps({
                "error": "Missing required query parameter: bucket"
            })
        }

    key_condition = Key("bucket").eq(bucket)

    if prefix:
        key_condition = key_condition & Key("object_key").begins_with(prefix)

    params = {
        "KeyConditionExpression": key_condition
    }

    if limit_raw:
        try:
            params["Limit"] = int(limit_raw)
        except ValueError:
            return {
                "statusCode": 400,
                "headers": {"Content-Type": "application/json"},
                "body": json.dumps({
                    "error": "limit must be an integer"
                })
            }

    response = table.query(**params)

    items = to_json_safe(response.get("Items", []))
    result = {
        "count": len(items),
        "items": items
    }

    if "LastEvaluatedKey" in response:
        result["next_start_key"] = to_json_safe(response["LastEvaluatedKey"])

    return {
        "statusCode": 200,
        "headers": {"Content-Type": "application/json"},
        "body": json.dumps(result)
    }

---

Part 2: Give the Function Permission to Query DynamoDB

This function needs dynamodb:Query permission on the upload_metadata table. A minimal policy looks like this:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "QueryMetadataTable",
      "Effect": "Allow",
      "Action": ["dynamodb:Query"],
      "Resource": "arn:aws:dynamodb:REGION:ACCOUNT_ID:table/upload_metadata"
    },
    {
      "Sid": "WriteLogs",
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "*"
    }
  ]
}

---

Part 3: Create the API Route

In API Gateway HTTP API:

1. Add a new route: GET /metadata/list
2. Integrate it with the new Lambda function
3. Deploy the API

---

Part 4: Test the Endpoint

Try one of these:

• GET /metadata/list?bucket=YOUR_BUCKET
• GET /metadata/list?bucket=YOUR_BUCKET&prefix=incoming/
• GET /metadata/list?bucket=YOUR_BUCKET&prefix=incoming/&limit=5

---

Part 5: Understand Prefix Filtering

Because object_key is the sort key, you can ask DynamoDB for everything in the bucket or only keys that begin with something like incoming/. This allows folder-like grouping to work naturally for a list API.

---

Note on Pagination

If LastEvaluatedKey appears in the response, there may be more items. You would use that key to continue the query on the next request. For this lesson, it is enough to return it in the API response so you can see that pagination exists.

---

Lab Checklist

---

Micro-activity 1: Response Review

---

Micro-activity 2: Query vs Scan

---

Summary

In this lesson, you turned a single-record read API into a multi-record list API. Because the table uses bucket as the partition key and object_key as the sort key, Query plus optional begins_with is the natural access pattern.

---

Quiz