dummy-link

AWSLambda

AWS Lambda interface for Julia

Readme

AWSLambda

AWS Lambda Interface for Julia

Build Status

Start by creating a basic AWSCore configuration...

using AWSCore
using AWSLambda
aws = aws_config(region = "us-east-1")

Deploy the base jl_lambda_eval function...

deploy_jl_lambda_base(aws)

Execute a function on Lambda...

@lambda_call(aws, readdir)("/var/task/bin")

1-element Array{ByteString,1}:
 "julia"

@lambda_call(aws, sum)([1,2,3])

6

Evaluate an expression on Lambda...

@lambda_eval aws ENV["LAMBDA_TASK_ROOT"]

"/var/task"

@lambda_eval aws run(`ls -l /var/task/bin/julia`)

-rwxr-xr-x 1 slicer 497 41748 Apr 20 05:33 /var/task/bin/julia

Evaluate an expression on Lambda...

@lambda_eval aws begin
    l = open(readlines, "/proc/cpuinfo")
    l = filter(i->ismatch(r"^model name", i), l)
    strip(split(l[1], ":")[2])
end

"Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz"

Deploy a Lambda function to count prime numbers...

λ = @λ aws function count_primes(low::Int, high::Int)
    count = length(primes(low, high))
    println("$count primes between $low and $high.")
    return count
end

The @λ macro creates an AWS Lambda named "count_primes". It wraps the body of the function with serialisation/deserialistion code and packages it into a .ZIP file. The .ZIP file deployed to AWS. The @λ macro returns an anonymous function that can be called to invoke the Lambda.

Run 20 instances of λ in parallel using amap()...

function count_primes(low::Int, high::Int)
    w = 500000000
    counts = amap(λ, [(i, min(high,i + w)) for i = low:w:high])
    count = sum(counts)
    println("$count primes between $low and $high.")
    return count
end

@test count_primes(10, 10000000000) == 455052507

Publish a verion with an alias...

lambda_publish_version(aws, "count_primes", "PROD")

Call a deployed Lambda by name with named arguments...

r = invoke_lambda(aws, "count_primes", low = 10, high = 100)
@test r[:jl_data] == "21"

Arguments and result are transfered as JSON.

Call a deployed Lambda and don't wait for the result...

r = async_lambda(aws, "count_primes", @SymDict(low = 10, high = 100))

Arguments are transfered as JSON.

Call a deployed Lambda by name...

r = invoke_jl_lambda(aws, "count_primes", 10 = 100)
@test r == 21

Arguments and result are transfered as serialised Julia objects.

Create a local module TestModule/TestModule.jl...

module TestModule

export test_function

__precompile__()

test_function(x) = x * x

end

Use the module in a Lambda...

push!(LOAD_PATH, "TestModule")

λ = @λ aws function lambda_test(x)

    # Check that precompile cache is being used...
    @assert !Base.stale_cachefile("/var/task/TestModule/TestModule.jl",
                                  Base.LOAD_CACHE_PATH[1] * /TestModule.ji")
    using TestModule
    return test_function(x)
end

@test λ(4) == 16

The @λ macro sees the `julia-observer-quote-cut-paste-4workstatement and bundles the correspondingjulia-observer-quote-cut-paste-5work` files into the deployment .ZIP file. It then does a dry-run invocation to trigger module precompilation. The resulting '.ji' files are retrieved from the Lambda sandbox and added to the deployment .ZIP file. Subsequent calls do not have to wait for compilation.

First Commit

12/30/2015

Last Touched

5 days ago

Commits

145 commits

Used By: