Note that AWS will very likely improve these numbers - they have a great track record of continuously delivering on such things. the NSRL hashsets, Videos, ML training sets, etc.). Good, but not enough for moving some interesting things (e.g. The effective bandwidth over this range of files sizes varied from 400 to 700 million bits per second. The bottom line here is that files larger than a several GB won’t reliably download in a single Lambda invocation. The image below shows the result of a recent one where a Step Function state machine is used to measure the time to download increasingly large files.Īttempting to download a set of increasingly large files. I’ve done some experiments to demonstrate the effective size of file that can be moved through a Lambda in this way. Specifically, this might mean getting more CPU cycles in less time, more bytes over the network in less time, more memory, etc.Īn example I like to use here is moving a large file into S3, where there will be a limit on the bandwidth available to the Function *and* a limit on the time the function can run (5 minutes). This is, of course, horizontal scaling (also known as “scaling out”) and works by using many resources to side-step limitations associated with a single resource. Fanout is a category of patterns for spreading work among multiple Function invocations to get more done sooner. Many of the most valuable uses I’ve found for Lambda involve cost and performance as core requirements - that is, if the service can be 10x faster or cheaper it will provide disruptive benefits to the customer.įanout is a key mechanism for achieving that kind of cost-efficient performance with Lambda. I have a love for FaaS, and in particular AWS Lambda for breaking so much ground in this space.
Using AWS Step Functions and Lambda for Fanout
0 kommentar(er)
