2022 W07


There’s been a gap of a week since my last set of notes; I was in Austria skiing! This week was a slow ramp up back to normal life and I think I’m finally there again.

Among some of the things I’ve been thinking about this week are: permissions structures in graph databases, infuriating AWS support practices, the role of developers as designers in early-stage start-ups, alpineJS, open contracting data standards. I won’t cover them all as a lot of the thoughts are very nascent but here are a few ramblings that seemed worth writing down…

Open Contracting Data

Open Contracting aims to improve the state of public-sector contracts so that they become more transparent, measurable and accountable (amongst other ideals). This can only be a good thing and it’s been nice to begin to get involved with this in the Health & Social Care sector in the UK. There’s a nice write-up of the goals of the project I’m involved in.

The standard itself is quite heavyweight and I’m yet to be convinced that it isn’t just a lot of time spent publishing to a specific spec with little involvement in how that spec is found/consumed. That said, there seems to be a good community actively developing tools to help with this and it’s all pretty early days so I’d love to be shown “the right way” that’s evolving here.

AWS ECS Task State Change Monitoring

We had a couple of occasions this week where our critical application tasks running on ECS were terminated. This had a knock-on effect on some of our other services and caused some outage that we didn’t want. I’m still yet to understand why the tasks were terminated (this information is only retained for an hour) but in the mean time, I can at least get ahead of the current game by monitoring when certain tasks stop.

This was surprisngly easy in AWS! You can use AWS EventBridge to send events published by AWS services for free. These can be piped into CloudWatch, incurring the regular cost per log size that CloudWatch always bills for.

For this first proof-of-concept, I used the UI and set it all up with the default selections (default event bus, log group names etc.) I set up a rule to pipe ECS state changes per cluster (staging/live) into cloudwatch log groups.

I can set up metrics in CloudWatch and produce alerts based on those metrics e.g. whenever a particular thing is stopped, whenever the rate of starts/stops is too high etc. Here’s an example of a filter pattern that I’m using for the important services

{ ($.detail.group = "service:my-service") && ($.detail.lastStatus = "STOPPED") }
© neverstew 2022
GitHub logo