Migrate Amplify blog source from BitBucket to GitHub

My blog remains hosted on Amplify for now. I may change that later.

I store the source code in BitBucket. My pushes to BitBucket stopped working around 2023-06-20 when BitbUcket replaced its SSH host key. I’ve waited until now to do something about it!

Now I use GitHub for all new projects, public and private. So rather than fix up the BitBucket configuration, I’ll migrate my blog source from BitBucket to GitHub.

The Amplify Hosting user interface doesn’t gives you an option to change this, and don’t see it in the documentation. Someone did already ask the same question in the amplify-hosting GitHub Repo, where a “ghost” (deleted GitHub user) shared a hint to make the change via the CLI using the update-app command:

aws amplify update-app --app-id <value> --repository <value> --access-token <GitHub PAT>

So I need:

  • A GitHub repository
  • an access token
  • a repository name (the URL?)
  • an app ID

Push the repository to GitHub

Change directory to the local copy of the repo.

Use the GitHub website to create a new empty repository.

Add a new remote for GitHub and push the repo.

git remote add github https://github.com/iainelder/isme-blog-amplify.git
git push github

For now there are two remotes. The default origin still points to BitBucket.

$ git remote -v
github	https://github.com/iainelder/isme-blog-amplify.git (fetch)
github	https://github.com/iainelder/isme-blog-amplify.git (push)
origin	git@bitbucket.org:isme/isme-blog-amplify.git (fetch)
origin	git@bitbucket.org:isme/isme-blog-amplify.git (push)

Create an access token

In GitHub create a new fine-grained personal access token with these settings:

  • Token name: isme-blog-amplify
  • Expiration: 2024-09-10
  • Description: Deploy to Amplify Hosting
  • Resource owner: iainelder
  • Repository access: Only select repositories
  • Selected repository: iainelder/isme-blog-amplify

  • Repository permissions:

    • Contents: read-only
    • Metadata: read-only
  • Account permissions: None

Save the token in 1Password as “Blog GitHub Personal Access Token”.

Write the token to a temporary file.

tmp="$(mktempdir blog)"
vim "$tmp/access-token.txt"

Get the repository name

I think that’s just the URL, so https://github.com/iainelder/isme-blog-amplify.

Get the app ID

Set the shell’s AWS session to the account region that hosts the the Amplify app.

List the apps by name and ID. This will give one JSONL record per app.

aws amplify list-apps | jq -c '.apps[] | {appId,name}'

{"appId":"d2skgb6grzx5i6","name":"isme-blog-amplify"}

Update the source repository

Put all the pieces together in this command:

aws amplify update-app \
--app-id  d2skgb6grzx5i6 \
--repository https://github.com/iainelder/isme-blog-amplify \
--access-token "file://$tmp/access-token.txt"

I get an error that apparently comes from GitHub to say that the personal access token (PAT) doesn’t grant access to the repository.

An error occurred (BadRequestException) when calling the UpdateApp operation: There was an issue setting up your repository. Please try again later.({"message":"Resource not accessible by personal access token","documentation_url":"https://docs.github.com/rest/webhooks/repos#create-a-repository-webhook"})

The linked document Create a repository webhook suggests that I need to allow the PAT to use the webhook API.

Add repository permissions:

  • Webhooks: Access: Read and write

Try again. The new response shows the new repository URL and new update time.

{
    "app": {
        "appId": "d2skgb6grzx5i6",
        "appArn": "arn:aws:amplify:eu-west-1:280507878471:apps/d2skgb6grzx5i6",
        "name": "isme-blog-amplify",
        "tags": {},
        "repository": "https://github.com/iainelder/isme-blog-amplify",
        "platform": "WEB",
        "createTime": "2021-01-15T13:41:49.628000+01:00",
        "updateTime": "2023-09-10T20:27:39.835000+02:00",
        "environmentVariables": {},
        "defaultDomain": "d2skgb6grzx5i6.amplifyapp.com",
        "enableBranchAutoBuild": false,
        "enableBranchAutoDeletion": false,
        "enableBasicAuth": false,
        "customRules": [
            {
                "source": "https://isme.es",
                "target": "https://www.isme.es",
                "status": "302"
            },
            {
                "source": "/<*>",
                "target": "/index.html",
                "status": "404-200"
            },
            {
                "source": "/linkedin",
                "target": "https://www.linkedin.com/in/isme-devops/",
                "status": "302"
            }
        ],
        "productionBranch": {
            "lastDeployTime": "2023-06-13T17:18:13.224000+02:00",
            "status": "SUCCEED",
            "branchName": "master"
        },
        "buildSpec": "# See repo",
        "customHeaders": "",
        "enableAutoBranchCreation": false,
        "repositoryCloneMethod": "TOKEN"
    }
}

I also see the new source repository in the Amplify console in Repository settings of the app. So it worked!

Deploy from the new repo

The change doesn’t appear to trigger a deployment.

Push a minor edit to the repo. This changes the status of the app to “running” in the Amplify console.

The deployment fails at the build step.

To show this via the API, get the job ID from the the console as the “Build X” number or from the ListJobs API, and call the GetJob API.

aws amplify get-job --app-id d2skgb6grzx5i6 --branch-name master --job-id 213 \
> "$tmp/job.json"

Hide the long URLs in the output using sed or using jq. (I show the sed way because I almost didn’t figure out the jq way.) This isn’t to redact as much as it is to truncate.

cat "$tmp/job.json" | sed 's|"https://.\+\?"|"https://..."|g' | jq

cat "$tmp/job.json" | jq '.. |= (strings |= sub("^https://.*$"; "https://..."))'

{
  "job": {
    "summary": {
      "jobArn": "arn:aws:amplify:eu-west-1:280507878471:apps/d2skgb6grzx5i6/branches/master/jobs/0000000213",
      "jobId": "213",
      "commitId": "f169571c3cf010ad00c834ba13825d670ac35d29",
      "commitMessage": "Add tools",
      "commitTime": "2023-09-10T20:51:45+02:00",
      "startTime": "2023-09-10T20:52:07.271000+02:00",
      "status": "FAILED",
      "endTime": "2023-09-10T20:53:07.157000+02:00"
    },
    "steps": [
      {
        "stepName": "BUILD",
        "startTime": "2023-09-10T20:52:07.338000+02:00",
        "status": "FAILED",
        "endTime": "2023-09-10T20:53:07.060000+02:00",
        "logUrl": "https://...",
        "artifactsUrl": "https://...",
        "testArtifactsUrl": "https://...",
        "testConfigUrl": "https://...",
        "context": "Image: AWS Amplify Console Official"
      },
      {
        "stepName": "DEPLOY",
        "status": "CANCELLED",
        "logUrl": "https://...",
        "artifactsUrl": "https://..."
      },
      {
        "stepName": "VERIFY",
        "status": "CANCELLED",
        "artifactsUrl": "https://...",
        "screenshots": {
          "googlepixel": "https://...",
          "ipadair2": "https://...",
          "iphone7plus": "https://...",
          "iphone8": "https://...",
          "samsungs7": "https://..."
        }
      }
    ]
  }
}

Show the build log.

cat "$tmp/job.json" \
| jq -r '.job.steps[] | select(.stepName == "BUILD") | .logUrl' \
| xargs curl

2023-09-10T18:52:31.803Z [ERROR]: !!! Unable to assume specified IAM Role. Please ensure the selected IAM Role has sufficient permissions and the Trust Relationship is configured correctly.
2023-09-10T18:52:31.876Z [INFO]: # Starting environment caching...
2023-09-10T18:52:32.244Z [INFO]: # Environment caching completed
Terminating logging...

The console also shows this message:

There was an issue connecting to your repo provider, click “Reconnect repository” in General Settings, and then try your build again.

When I “Reconnect repository” I see this over a button labelled “Configure GitHub App”.

Install and authorize GitHub App. Authorize the AWS Amplify app to your GitHub account and Install it for the repos you want to connect to Amplify.

So it seems that the webhook either is only part of the solution, or it is obsolete.

The Amplify Hosting document history shows that the GitHub app apeared in the documentation on 2022-04-05, a year after I set up the BitBucket integration.

Read Setting up Amplify access to GitHub repositories. “Existing Amplify apps that you previously connected from GitHub repositories use OAuth for repo access.” “Migration must be performed in the Amplify console in the AWS Management Console.”

Click “Configure GitHub App”.

AWS Amplify (eu-west-1) by AWS Amplify Console would like permission to:

  • Verify your GitHub identity (iainelder)
  • Know which resources you can access
  • Act on your behalf

Click “Authorize AWS Amplify (eu-west-1)”.

Install & Authorize AWS Amplify (eu-west-1)

Install & Authorize on your personal account Iain Samuel McLean Elder

Choose “Only select repositories”.

Select iainelder/isme-blog-amplify.

with these permissions:

  • Write access to files located at amplify.yml
  • Read access to code and metadata
  • Read and write access to checks, pull requests, and repository hooks

Click “Install & Authorize”.

It redirects me to the Amplify console, which shows these error messages:

  • GitHub authorization failed
  • We were not able to authorize you with GitHub.

I click “Cancel” and click “Configure GitHub App” again. This time it says “GitHub authorization successful”.

Click Next.

Once you click ‘Complete installation’ Amplify will delete your old webhook and create a new one using our GitHub App.

Click “Complete installation”.

The console shows these success messages:

  • Updating app: isme-blog-amplify success
  • Your app was successfully reconnected with GitHub!

Navigate to build 213 in the Amplify console and click “Redeploy this version”.

Watch the deployment in the console. It shows the message “Build successfully completed”. The deploy stage also complete.

The CloudWatch event for the deployment shows that I could have used the StartJob API to redeploy.

{
    "requestParameters": {
        "jobId": "213",
        "commitMessage": "Add tools",
        "commitTime": 1694371905,
        "appId": "d2skgb6grzx5i6",
        "branchName": "master",
        "commitId": "f169571c3cf010ad00c834ba13825d670ac35d29",
        "jobType": "RETRY"
    },
    "responseElements": {
        "jobSummary": {
            "commitId": "f169571c3cf010ad00c834ba13825d670ac35d29",
            "commitMessage": "Add tools",
            "commitTime": 1694371905,
            "jobArn": "arn:aws:amplify:eu-west-1:280507878471:apps/d2skgb6grzx5i6/branches/master/jobs/0000000214",
            "jobId": "214",
            "status": "PENDING"
        }
    }
}

Check the production site for the update. It works!

Move the local folder

A final bit of housekeeping: stop using the remote name as part of my repo hierarchy. A repo can have multiple remotes, so just organize by function or project instead.

cd ~/Repos
mkdir isme
mv Personal/bitbucket/isme-blog-amplify/ isme/blog
cd ~/Repos/isme/blog

Finally remove the BitBucket remote. We’re going all-in on GitHub.

git remote remove origin
git remote rename gitub origin

$ git remote -v
origin	https://github.com/iainelder/isme-blog-amplify.git (fetch)
origin	https://github.com/iainelder/isme-blog-amplify.git (push)

Done!