Getting Your Medium Posts in JSON

3 min readNov 23, 2020

How to use AWS CDK to transform the Medium RSS feed into JSON.

Medium might have a JSON endpoint, but it does not have all the story content like the RSS feed has. Let me show you what I have so far, as this is a problem I’ve tried to solve before:

The issue with this approach is that the latency for getting our desired posts would be within the 3–7 seconds range because we’re also making a request to medium for the RSS posts, then transforming those into JSON, and finally returning those to the user. This would happen on every request. I tried mitigating this with cache from API Gateway, but it was around $10/month for up to an hour of cache. Since my website doesn’t get much traffic, that doesn’t really make sense (the cache probably won’t be hit too often.)

Therefore, I wanted to change this so that we still get post updates, but more importantly we get a longer cache of our JSON asset. My goal was to use CloudFront caching since this is where my website is also hosted. I could also update the JSON once or twice a day, since I don’t really post that often. This made me think of the following architecture:

The bottom part looks a bit complicated, but really its not. It is also reusable—meaning if someone else wanted to implement the same thing, the only difference is which medium account they want to target. For this reason, I decided to abstract this out into its own CDK Construct: the MediumCDKConstruct

With this construct, you define your medium user name, and optionally they key of the JSON object and Schedule to update the JSON object, and you get the shaded part of the diagram. Then you just use the JSON asset however you want: on a website, app, or standalone.

To be able to re-use this infrastructure you just need an AWS account. Then you can follow the tutorial to get started on creating a construct, and you can use MediumCDKConstruct after installing from NPM.

Then, you just need to point an origin from CloudFront (leave the path blank) and then a behavior so that ‘medium/posts’ goes to that origin (or whatever S3 key your JSON is at.) This is what using this construct looks like for my website:

import { Construct, Stack, StackProps } from '@aws-cdk/core';
import { MediumCdkConstruct } from '@canahui/medium-posts';export class MediumCdkStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);    new MediumCdkConstruct(this, 'JoseMediumBlogPostsJSON', {
      mediumUser: '@josecanahui'
    })
  }
}

And just like that, an S3 bucket was created, along with a lambda that executes twice a day to transform my posts into JSON. I can now access my Medium posts client-side from my react website way faster: https://www.canahui.me

The JSON of these posts also has the content of the blog posts, which is nice if you want to render your posts in your own website (something the medium JSON API does not do.)

Getting Your Medium Posts in JSON

Written by Jose Canahui