My multi processing needs are very simple: I work in machine learning, and I sometimes need to evaluate an algorithm in multiple datasets, or multiple algorithms in a dataset, or some such. I just need to run a function with some arguments and get a number.

I need no RPC, shared data, nothing.

In Julia, I am getting an error with the following code:

type Model
    param
end

# 1. I have several algorithms/models
models = [Model(i) for i in 1:50]

# 2. I have one dataset
X = rand(50, 5)

# 3. I want to paralelize this function
@everywhere function transform(m)
    sum(X .* m.param)
end

addprocs(3)
println(pmap(transform, models))

I keep getting errors such as,

ERROR: LoadError: On worker 2:
UndefVarError: #transform not defined

Also, is there a way to avoid having to place @everywhere everywhere? Can I just tell that all variables should be copied over to the workers when they are created (as is done in Python multiprocessing)?

My typical code looks obviously much more complicated than this, with models ranging several files.

For reference, this is what I would do in Python:

import numpy as np
import time

# 1. I have several algorithms/models
class Model:
    def __init__(self, param):
        self.param = param
models = [Model(i) for i in range(1,51)]

# 2. I have one dataset
X = np.random.random((50, 5))

# 3. I want to paralelize this function
def transform(m):
    return np.sum(X * m.param)

import multiprocessing
pool = multiprocessing.Pool(4)
print(pool.map(transform, models))

I could not find the option to enable bracket and quote autocomplete option in Julia, and the IDE I use is Atom. Anyone knows how to do it?

My question is simple, is there a generic Float type in julia? For the case of Integers, for example, one can simply write Int and it will be translated as Int32 in 32-bit systems or Int64 in 64-bit systems. For Floats though, see the example function below:

function bb(n)
    b = Array{Float64}(n) 
    b[1] = 0.9999
    for i = 2:n
        @inbounds b[i] = b[i-1] * 0.9999
    end    
    println(b[n])
end

bb(10^3)
@time bb(10^3)
@time bb(10^8)

It gives the following timing results along with total memory allocations:

0.9048328935585562
0.9048328935585562
  0.000100 seconds (135 allocations: 15.750 KB)
2.4703e-320
  3.230642 seconds (14 allocations: 762.940 MB, 1.51% gc time)

Now change the first line to b = Array{AbstractFloat}(n) and see the ridiculously huge timings and memory allocations:

0.9048328935585562
0.9048328935585562
  0.003564 seconds (2.13 k allocations: 46.953 KB)
2.4703e-320
  351.068176 seconds (200.00 M allocations: 3.725 GB, 0.74% gc time)

There is nothing as b = Array{Float}(n) I can use, the only solution I came up with is this non-elegant notation b = Array{typeof(1.0)}(n).

I'm using LightGraphs.jl package to create some graphs. I have a program which runs in many iterations. I need to draw the graph for each iteration and save each of them in separate files. Can someone please help me to do this?

Thanks,

I'm trying to use JuMP to solve a non-linear problem, where the number of variables are decided by the user - that is, not known at compile time.

To accomplish this, the @NLobjective line looks like this:

@eval @JuMP.NLobjective(m, Min, $(Expr(:call, :myf, [Expr(:ref, :x, i) for i=1:n]...)))

Where, for instance, if n=3, the compiler interprets the line as identical to:

@JuMP.NLobjective(m, Min, myf(x[1], x[2], x[3]))

The issue is that @eval works only in the global scope, and when contained in a function, an error is thrown.

My question is: how can I accomplish this same functionality -- getting @NLobjective to call myf with a variable number of x[1],...,x[n] arguments -- within the local, not-known-at-compilation scope of a function?

def testme(n)
    myf(a...) = sum(collect(a).^2)

    m = JuMP.Model(solver=Ipopt.IpoptSolver())

    JuMP.register(m, :myf, n, myf, autodiff=true)
    @JuMP.variable(m, x[1:n] >= 0.5)

    @eval @JuMP.NLobjective(m, Min, $(Expr(:call, :myf, [Expr(:ref, :x, i) for i=1:n]...)))
    JuMP.solve(m)
end

testme(3)

Thanks!

I want to declare a vector of vector in Julia like the following

V = [v1, v2, v3,...] where v1, v2, v3... have dimension of K x 1

What is the syntax to achieve this?

How can I create a DataFrame with Julia with index names that are different from Row and write values into a (index,column) pair?

I do the following in Python with pandas:

import pandas as pd
df = pd.DataFrame(index = ['Maria', 'John'], columns = ['consumption','age'])
df.loc['Maria']['age'] = 52 

I would like to do the same in Julia. How can I do this? The documentation shows a DataFrame similar to the one I would like to construct but I cannot figure out how.

I have to compute the probability of Poisson distribution in Julia. I just know how to get Poisson distribution. But i have to compute the probability. also i have lambda from 20 to 100.

using Distributions
Poisson()

I was wondering if Julia has a package similar to Sparklyr in R that could handle out of memory data. My data is 11 GB in csv format.

I installed HPAT package in Julia, but I am not sure if it helps to handle big data. In addition, I noticed that there is a Spark package in Julia, does it have any function that enables me to import local data, like the spark_read_csv function in Sparklyr?

I just want to know how to convert an array{Float64,1} to a Float64? in julia

utility= rand(1)*row*c*(1-x)

it gives me error "array of size (1,) passed as objective; only scalar objectives are allowed"

EDIT: In the end it had nothing to do with it. I'm answering for anyone in a similar situation.

I discovered Julia a month ago and have been fascinated with it since then. However, it seems compilation (or a variable scope) is still not clear for me.

I have this module in plots.jl:

module plottinghhkp

instance_path = "somestring"

function dosomething()
  ...
end

...
end

and then include it in the REPL to run module functions

julia> include("plots.jl")
...
julia> hhkplot.instance_path
julia> "somestring"

I then modify instance_path, and re-import using include again, but instance_path shows old value instead of the new value:

julia> include("plots.jl")
...
julia> plottinghhkp.instance_path
"somestring"

My solution was to rename the variable to instancepath so that a new symbol is created, however, autocomplete now shows:

julia> plottinghhkp.
instance_path  instancepath
julia> plottinghhkp.instancepath
"newvalue"
julia> plottinghhkp.instance_path
"somestring"

when the instance_path symbol does not exist anymore in the module. Creating a fresh environment with workspace() never ends.

Why is this happening? Do I need to explicitly load Julia with no precompiling, or add __precompile__(false) to the module? Or is it something about the variable scopes?

Thanks.

How can we reserve memory (or allocate memory without initialization) in Julia? In C++, a common pattern is to call reserve before calling push_back several times to avoid having to call on malloc more than once. Is there an equivalent in Julia?

I'm using Atom and Julia to plot a good number of graphs. I have a code like this:

function plotnetwork(A)
  for i=1:size(A,3)
    p,t=findn(A[:,:,i]
    graphplot(p,t)
  end
return
end

where graphplot is grom PlotRecipes: https://github.com/JuliaPlots/PlotRecipes.jl

I have the following problems: using this code as a function doesn't plot anything (like if the plots where local variables) and even putting this loop in my main only outputs a single plot in a single window.

What I desire is multiple windows with one plot each, I think the command push may be what I need but I didn't find much online.

I am working on a program that a friend wrote in Julia. We have to do some work for university, especially for a course called "dynamic modelling".

The thing is to reproduce some great work of Kashtan and Alon; maybe you know their publication about biological networks and genetic algorithms. Now my friend asked me to find out, whether we can plot something in 3D and I don't know how to do that. He gave me some part of our program (see code below) and wrote me to find out, whether we can draw a line from one coordinate (x,y,z) to another coordinate (x,y,z). Do you know how to do that in Julia?

 module ShowGateNet

export
    showGate,
    setGattaStruct;

using Plots;
using BPIPopulation;

abstract GrafGate

type GGate <: GrafGate
    in1::Array{Integer};
    in2::Array{Integer};
    out::Array{Integer};
end;

#---------------------------------------------------------------
# creates a 2D-matrix for almost a circle:
# mx:: x-coordinate of the center circle
# my:: y-coordinate of the center circle
#  r:: radius of the circle
#---------------------------------------------------------------
 function circNumba(rx=2::Integer,cx=0::Integer,cy=0::Integer, cz=0)
   k=0:360;
   c=zeros(length(k),4);
   c[:,1] = k;
   c[:,2] = round(cos(radian(k))*rx+cx,3);
   c[:,3] = cy;
    c[:,4] = round(sin(radian(k))*rx+cz,3);
   return c;     
 end;


function radian(grad) 
   return round(2*pi*grad/360,3);
end

function showGate(cx,cy,cz,numba)
    r = 2;
    centr = [cx;cy;cz];
    # A = [centr[1]-1;centr[2];centr[3]];
    # B = [centr[1];centr[2]+1;centr[3]];
    C = [centr[1];centr[2]-1;centr[3]];
    in2 = [centr[1]-3;centr[2]+1;centr[3]-1];
    in1 = [centr[1]+3;centr[2]-1;centr[3]-1];
    K = [centr[1];centr[2];centr[3]-1];
    O = [centr[1];centr[2];centr[3]-3];
    kCirc = circNumba(r,K[1],K[2],K[3]);
    #display(plot!(kCirc[:,2],kCirc[:,3],kCirc[:,4],title="test-circ",line=3,color="red",zlabel="Z",xlabel="X",ylabel="Y"));
    #display(plot!([centr[1],centr[1],centr[1]],[centr[2],centr[2],centr[2]],[centr[3]-1-r,centr[3]-1-r,centr[3]-3-r],w=3,color="blue"));
    #display(plot!([centr[1]-3,centr[1]-3,centr[1]-2],[centr[2],centr[2],centr[2]],[centr[3],centr[3],centr[3]]-1,w=3,color="green")); #A nach K
    #display(plot!([centr[1]+2,centr[1]+2,centr[1]+3],[centr[2],centr[2],centr[2]],[centr[3],centr[3],centr[3]]-1,w=3,color="green")); #A nach K    
    display(plot(kCirc[:,2],kCirc[:,3],kCirc[:,4],title="test-circ",line=3,color="red",zlabel="Z",xlabel="X",ylabel="Y"));
    plot!([centr[1],centr[1],centr[1]],[centr[2],centr[2],centr[2]],[centr[3]-1-r,centr[3]-1-r,centr[3]-3-r],w=3,color="blue");#senkrechte
    plot!([centr[1]-3,centr[1]-3,centr[1]-2],[centr[2],centr[2],centr[2]],[centr[3],centr[3],centr[3]]-1,w=3,color="green"); #A nach K
    plot!([centr[1]+2,centr[1]+2,centr[1]+3],[centr[2],centr[2],centr[2]],[centr[3],centr[3],centr[3]]-1,w=3,color="green"); #A nach K
    plotNumba(K,numba);
end;

function showGate2(cx,cy,cz,numba)
    r = 2;
    centr = [cx;cy;cz];
    # A = [centr[1]-1;centr[2];centr[3]];
    # B = [centr[1];centr[2]+1;centr[3]];
    C = [centr[1];centr[2]-1;centr[3]];
    in2 = [centr[1]+1;centr[2]+1;centr[3]];
    in1 = [centr[1]+1;centr[2]-1;centr[3]];
    K = [centr[1];centr[2]-r*2;centr[3]-1];
    O = [centr[1];centr[2];centr[3]-1];
    kCirc = circNumba(r,K[1],K[2],K[3]);
    display(plot(kCirc[:,2],kCirc[:,3],kCirc[:,4],title="test-circ",line=3,color="red",zlabel="Z",xlabel="X",ylabel="Y"));
    plot!([centr[1],centr[1],centr[1]],[centr[2],centr[2],centr[2]],[centr[3]-1-r,centr[3]-1-r,centr[3]-3-r],w=3,color="blue");
    plotNumba(K,numba);
end;


function showGate2!(cx,cy,cz,numba)
    r = 2;
    centr = [cx;cy;cz];
    # A = [centr[1]-1;centr[2];centr[3]];
    # B = [centr[1];centr[2]+1;centr[3]];
    C = [centr[1];centr[2]-1;centr[3]];
    in2 = [centr[1]+1;centr[2]+1;centr[3]];
    in1 = [centr[1]+1;centr[2]-1;centr[3]];
    K = [centr[1];centr[2];centr[3]-1];
    O = [centr[1];centr[2];centr[3]-1];
    kCirc = circNumba(r,K[1],K[2],K[3]);
    plot!(kCirc[:,2],kCirc[:,3],kCirc[:,4],title="test-circ",line=3,color="red",zlabel="Z",xlabel="X",ylabel="Y");
    plot!([centr[1],centr[1],centr[1]],[centr[2],centr[2],centr[2]],[centr[3]-1-r,centr[3]-1-r,centr[3]-3-r],w=3,color="blue");
    plot!([centr[1]-3,centr[1]-3,centr[1]-2],[centr[2],centr[2],centr[2]],[centr[3],centr[3],centr[3]]-1,w=3,color="green"); #A nach K
    plot!([centr[1]+2,centr[1]+2,centr[1]+3],[centr[2],centr[2],centr[2]],[centr[3],centr[3],centr[3]]-1,w=3,color="green"); #A nach K
    plotNumba(K,numba);
end;

function showGateIn!(cx,cy,cz,numba)
    r = 2;
    centr = [cx;cy;cz];
    # A = [centr[1]-1;centr[2];centr[3]];
    # B = [centr[1];centr[2]+1;centr[3]];
    C = [centr[1];centr[2]-1;centr[3]];
    in2 = [centr[1]+1;centr[2]+1;centr[3]];
    in1 = [centr[1]+1;centr[2]-1;centr[3]];
    K = [centr[1];centr[2];centr[3]-1];
    O = [centr[1];centr[2];centr[3]-1];
    kCirc = circNumba(r,K[1],K[2],K[3]);
    plot!(kCirc[:,2],kCirc[:,3],kCirc[:,4],title="test-circ",line=3,color="red",zlabel="Z",xlabel="X",ylabel="Y");
    if numba == 1
        display(plot!([centr[1],centr[1],centr[1]],[centr[2],centr[2],centr[2]],[centr[3]-1-r,centr[3]-1-r,centr[3]-3-r],w=3,color="blue"));
    else
        plot!([centr[1],centr[1],centr[1]],[centr[2],centr[2],centr[2]],[centr[3]-1-r,centr[3]-1-r,centr[3]-3-r],w=3,color="blue");
    end;
    plotNumba(K,numba);
end;



function showGateL!(cx,cy,cz,numba)
    r = 2;
    centr = [cx;cy;cz];
    # A = [centr[1]-1;centr[2];centr[3]];
    # B = [centr[1];centr[2]+1;centr[3]];
    C = [centr[1];centr[2]-1;centr[3]];
    in2 = [centr[1]+1;centr[2]+1;centr[3]];
    in1 = [centr[1]+1;centr[2]-1;centr[3]];
    K = [centr[1]-1-r;centr[2];centr[3]-1];
    O = [centr[1];centr[2];centr[3]-1];
    kCirc = circNumba(r,K[1],K[2],K[3]);
    plot!(kCirc[:,2],kCirc[:,3],kCirc[:,4],title="test-circ",line=3,color="red");
    plot!([centr[1]-1,centr[1]-1,centr[1]],[centr[2],centr[2],centr[2]],[centr[3],centr[3],centr[3]]-1,w=3,color="green"); #A nach K
    plot!([centr[1],centr[1],centr[1]+1],[centr[2],centr[2],centr[2]]+0.5,[centr[3]-1,centr[3]-1,centr[3]-1],w=3,color="green");# B nach D
    plot!([centr[1],centr[1],centr[1]+1],[centr[2],centr[2],centr[2]]-0.5,[centr[3],centr[3],centr[3]]-1,w=3,color="green");# C nach E
    plot!([centr[1],centr[1],centr[1]],[centr[2]-0.5,centr[2]-0.5,centr[2]+0.5],[centr[3],centr[3],centr[3]]-1,w=3,color="green"); # C nach B (waagerechte
    plot!([centr[1],centr[1],centr[1]],[centr[2],centr[2],centr[2]],[centr[3]-1,centr[3]-1,centr[3]-3],w=3,color="blue");
    plotNumba(K,numba);
end;

function showGateR!(cx,cy,cz,numba)
    r = 2;
    centr = [cx;cy;cz];
    # A = [centr[1]-1;centr[2];centr[3]];
    # B = [centr[1];centr[2]+1;centr[3]];
    C = [centr[1];centr[2]-1;centr[3]];
    in2 = [centr[1]+1;centr[2]+1;centr[3]];
    in1 = [centr[1]+1;centr[2]-1;centr[3]];
    K = [centr[1]+1+r;centr[2];centr[3]-1];
    O = [centr[1];centr[2];centr[3]-1];
    kCirc = circNumba(r,K[1],K[2],K[3]);
    plot!(kCirc[:,2],kCirc[:,3],kCirc[:,4],title="test-circ",line=3,color="red");
    plot!([centr[1]+1,centr[1]+1,centr[1]],[centr[2],centr[2],centr[2]],[centr[3],centr[3],centr[3]]-1,w=3,color="green"); #A nach K
    plot!([centr[1],centr[1],centr[1]-1],[centr[2],centr[2],centr[2]]+0.5,[centr[3]-1,centr[3]-1,centr[3]-1],w=3,color="green");# B nach D
    plot!([centr[1],centr[1],centr[1]-1],[centr[2],centr[2],centr[2]]-0.5,[centr[3],centr[3],centr[3]]-1,w=3,color="green");# C nach E
    plot!([centr[1],centr[1],centr[1]],[centr[2]-0.5,centr[2]-0.5,centr[2]+0.5],[centr[3],centr[3],centr[3]]-1,w=3,color="green"); # C nach B (waagerechte
    plot!([centr[1],centr[1],centr[1]],[centr[2],centr[2],centr[2]],[centr[3]-1,centr[3]-1,centr[3]-3],w=3,color="blue");
    plotNumba(K,numba);
end;

#function setGattaStruct(pEle::BPIPopulation.popElement)
function setGattaStruct()   
    showGate(0,0,-4,15);  # 15
    showGate2!(-4,0,8,14); #14
    showGate2!(4,0,8,13);   #13
    showGateL!(-8,4,16,12); #12
    showGateL!(-8,-4,16,11); #11
    showGateR!(8,4,16,10); #10
    showGateR!(8,-4,16,9); #9
    showGateL!(-8,4, 32,8); #8
    showGateL!(-8,-4, 32,7); #7
    showGateR!(8,4, 32,6); #6
    showGateR!(8,-4, 32,5); #5
    showGateIn!(0,6, 40,4); #4
    showGateIn!(0,2, 40,3); #3
    showGateIn!(0,-2, 40,2);    #2
    showGateIn!(0,-6, 40,1);    #1
    # hier dann evtl "PLOTTEN "der verbindungen 

end;


function plotNumba(K,numba)
    if numba>=10
        plot!([K[1],K[1],K[1]]-0.25,[K[2],K[2],K[2]],
            [K[3]-1,K[3]-1,K[3]-3]+2,w=3,color="blue"); 
    end;    
    if numba%10 == 0
        plot!([K[1],K[1],K[1]]+1.25,[K[2],K[2],K[2]],
            [K[3]-1,K[3]-1,K[3]-3]+2,w=3,color="blue"); 
        plot!([K[1],K[1],K[1]]+0.25,[K[2],K[2],K[2]],
            [K[3]-1,K[3]-1,K[3]-3]+2,w=3,color="blue"); 
        plot!([K[1]+1,K[1]+1,K[1]]+0.25,[K[2],K[2],K[2]],
            [K[3],K[3],K[3]]-1,w=3,color="blue"); 
        plot!([K[1]+1,K[1]+1,K[1]]+0.25,[K[2],K[2],K[2]],
            [K[3],K[3],K[3]]+1,w=3,color="blue"); 
    end;    
    if numba%10 == 1
        plot!([K[1],K[1],K[1]]+1.25,[K[2],K[2],K[2]],
            [K[3]-1,K[3]-1,K[3]-3]+2,w=3,color="blue"); 
    end;
    if numba%10 == 2
        plot!([K[1],K[1],K[1]]+0.25,[K[2],K[2],K[2]],
            [K[3]-2,K[3]-2,K[3]-3]+2,w=3,color="blue");
        plot!([K[1],K[1],K[1]]+1.25,[K[2],K[2],K[2]],
            [K[3]-1,K[3]-1,K[3]-2]+2,w=3,color="blue");
        plot!([K[1]+1,K[1]+1,K[1]]+0.25,[K[2],K[2],K[2]],
            [K[3],K[3],K[3]]-1,w=3,color="blue"); 
        plot!([K[1]+1,K[1]+1,K[1]]+0.25,[K[2],K[2],K[2]],
            [K[3],K[3],K[3]],w=3,color="blue"); 
        plot!([K[1]+1,K[1]+1,K[1]]+0.25,[K[2],K[2],K[2]],
            [K[3],K[3],K[3]]+1,w=3,color="blue"); 
    end;
    if numba%10 == 3
        plot!([K[1],K[1],K[1]]+1.25,[K[2],K[2],K[2]],
            [K[3]-1,K[3]-1,K[3]-3]+2,w=3,color="blue");
        plot!([K[1]+1,K[1]+1,K[1]]+0.25,[K[2],K[2],K[2]],
            [K[3],K[3],K[3]]-1,w=3,color="blue"); 
        plot!([K[1]+1,K[1]+1,K[1]]+0.25,[K[2],K[2],K[2]],
            [K[3],K[3],K[3]],w=3,color="blue"); 
        plot!([K[1]+1,K[1]+1,K[1]]+0.25,[K[2],K[2],K[2]],
            [K[3],K[3],K[3]]+1,w=3,color="blue");
    end;    
    if numba%10 == 4
        plot!([K[1],K[1],K[1]]+1.25,[K[2],K[2],K[2]],[K[3]-1,K[3]-1,K[3]-3]+2,w=3,color="blue");
        plot!([K[1]+1,K[1]+1,K[1]]+0.25,[K[2],K[2],K[2]],[K[3],K[3],K[3]],w=3,color="blue"); 
        plot!([K[1],K[1],K[1]]+0.25,[K[2],K[2],K[2]],[K[3]-1,K[3]-1,K[3]-2]+2,w=3,color="blue");
    end;    
    if numba%10 == 5
        plot!([K[1],K[1],K[1]]+1.25,[K[2],K[2],K[2]],[K[3]-2,K[3]-2,K[3]-3]+2,w=3,color="blue");
        plot!([K[1],K[1],K[1]]+0.25,[K[2],K[2],K[2]],[K[3]-1,K[3]-1,K[3]-2]+2,w=3,color="blue");
        plot!([K[1]+1,K[1]+1,K[1]]+0.25,[K[2],K[2],K[2]],[K[3],K[3],K[3]]-1,w=3,color="blue");
        plot!([K[1]+1,K[1]+1,K[1]]+0.25,[K[2],K[2],K[2]],[K[3],K[3],K[3]],w=3,color="blue"); 
        plot!([K[1]+1,K[1]+1,K[1]]+0.25,[K[2],K[2],K[2]],[K[3],K[3],K[3]]+1,w=3,color="blue"); 
    end;
   if numba%10 == 6
        plot!([K[1],K[1],K[1]]+1.25,[K[2],K[2],K[2]],[K[3]-2,K[3]-2,K[3]-3]+2,w=3,color="blue"); 
        plot!([K[1],K[1],K[1]]+0.25,[K[2],K[2],K[2]],[K[3]-1,K[3]-1,K[3]-3]+2,w=3,color="blue"); 
        plot!([K[1]+1,K[1]+1,K[1]]+0.25,[K[2],K[2],K[2]],[K[3],K[3],K[3]]-1,w=3,color="blue"); 
        plot!([K[1]+1,K[1]+1,K[1]]+0.25,[K[2],K[2],K[2]],[K[3],K[3],K[3]],w=3,color="blue"); 
        plot!([K[1]+1,K[1]+1,K[1]]+0.25,[K[2],K[2],K[2]],[K[3],K[3],K[3]]+1,w=3,color="blue");
    end;    
    if numba%10 == 7
        plot!([K[1],K[1],K[1]]+1.25,[K[2],K[2],K[2]],[K[3]-1,K[3]-1,K[3]-3]+2,w=3,color="blue"); 
        plot!([K[1]+1,K[1]+1,K[1]]+0.25,[K[2],K[2],K[2]],[K[3],K[3],K[3]]+1,w=3,color="blue"); 
    end;            
    if numba%10 == 8
        plot!([K[1],K[1],K[1]]+1.25,[K[2],K[2],K[2]],[K[3]-1,K[3]-1,K[3]-3]+2,w=3,color="blue");
        plot!([K[1],K[1],K[1]]+0.25,[K[2],K[2],K[2]],[K[3]-1,K[3]-1,K[3]-3]+2,w=3,color="blue");
        plot!([K[1]+1,K[1]+1,K[1]]+0.25,[K[2],K[2],K[2]],[K[3],K[3],K[3]]-1,w=3,color="blue"); 
        plot!([K[1]+1,K[1]+1,K[1]]+0.25,[K[2],K[2],K[2]],[K[3],K[3],K[3]],w=3,color="blue"); 
        plot!([K[1]+1,K[1]+1,K[1]]+0.25,[K[2],K[2],K[2]],[K[3],K[3],K[3]]+1,w=3,color="blue");
    end;    
     if numba%10 == 9
        plot!([K[1],K[1],K[1]]+0.25,[K[2],K[2],K[2]],[K[3]-1,K[3]-1,K[3]-2]+2,w=3,color="blue");
        plot!([K[1],K[1],K[1]]+1.25,[K[2],K[2],K[2]],[K[3]-1,K[3]-1,K[3]-3]+2,w=3,color="blue"); 
        plot!([K[1]+1,K[1]+1,K[1]]+0.25,[K[2],K[2],K[2]],[K[3],K[3],K[3]]-1,w=3,color="blue"); 
        plot!([K[1]+1,K[1]+1,K[1]]+0.25,[K[2],K[2],K[2]],[K[3],K[3],K[3]],w=3,color="blue"); 
        plot!([K[1]+1,K[1]+1,K[1]]+0.25,[K[2],K[2],K[2]],[K[3],K[3],K[3]]+1,w=3,color="blue");
    end;    
end;
end; # END MODULE

I wrote a little function to get the byte offset of a field inside inside a struct, but to accomplish that, I'm iterating each time over the fields till I found the symbol. In C however, the offsetof() is calculated only once at compile time, since the offsets do not change inside a struct anymore. I wonder how to optimize this as much as possible, since it's a very basic function, which I need to call very often for each frame (3D game engine).

function offsetof(type_, member::Symbol)
  for (i, item) in enumerate(fieldnames(type_))
    if item == member
      return fieldoffset(type_, i)
    end
    #print(typeof(i))
  end
  # what to do when symbol not in type_?
  throw("$type_ has no member named $member")
end

Usage:

type ABC
    a::Int64
    b::Int64
    c::Int64
end
offsetof(ABC, :a) # 0
offsetof(ABC, :b) # 8
offsetof(ABC, :c) # 16

Looking at the LLVM code, it's a lot of code:

@code_llvm offsetof(Model_s, :fov)

Output:

; Function Attrs: uwtable
define i64 @julia_offsetof_61750(i8**, i8**) #0 !dbg !5 {
top:
  %2 = call i8**** @jl_get_ptls_states() #6
  %3 = alloca [13 x i8**], align 8
  %.sub = getelementptr inbounds [13 x i8**], [13 x i8**]* %3, i64 0, i64 0
  %4 = getelementptr [13 x i8**], [13 x i8**]* %3, i64 0, i64 8
  %5 = getelementptr [13 x i8**], [13 x i8**]* %3, i64 0, i64 2
  %6 = getelementptr [13 x i8**], [13 x i8**]* %3, i64 0, i64 3
  %7 = getelementptr [13 x i8**], [13 x i8**]* %3, i64 0, i64 4
  %8 = getelementptr [13 x i8**], [13 x i8**]* %3, i64 0, i64 5
  %9 = getelementptr [13 x i8**], [13 x i8**]* %3, i64 0, i64 6
  %10 = getelementptr [13 x i8**], [13 x i8**]* %3, i64 0, i64 7
  %11 = bitcast i8*** %4 to i8*
  call void @llvm.memset.p0i8.i32(i8* %11, i8 0, i32 40, i32 8, i1 false)
  %12 = bitcast [13 x i8**]* %3 to i64*
  %13 = bitcast i8*** %5 to i8*
  call void @llvm.memset.p0i8.i64(i8* %13, i8 0, i64 40, i32 8, i1 false)
  store i64 22, i64* %12, align 8
  %14 = getelementptr [13 x i8**], [13 x i8**]* %3, i64 0, i64 1
  %15 = bitcast i8**** %2 to i64*
  %16 = load i64, i64* %15, align 8
  %17 = bitcast i8*** %14 to i64*
  store i64 %16, i64* %17, align 8
  store i8*** %.sub, i8**** %2, align 8
  store i8** null, i8*** %10, align 8
  %18 = getelementptr [13 x i8**], [13 x i8**]* %3, i64 0, i64 12
  %19 = getelementptr [13 x i8**], [13 x i8**]* %3, i64 0, i64 11
  %20 = getelementptr [13 x i8**], [13 x i8**]* %3, i64 0, i64 10
  %21 = getelementptr [13 x i8**], [13 x i8**]* %3, i64 0, i64 9
  %22 = call i8** @jlsys_fieldnames_43495(i8** inttoptr (i64 375726352 to i8**))
  store i8** %22, i8*** %5, align 8
  %23 = getelementptr inbounds i8*, i8** %22, i64 1
  %24 = bitcast i8** %23 to i64*
  %25 = getelementptr i8*, i8** %22, i64 3
  %26 = bitcast i8** %25 to i64*
  %27 = bitcast i8** %22 to i8****
  %28 = bitcast i8**** %2 to i8*
  br label %L3

L3:                                               ; preds = %cont2, %top
  %"#temp#1.sroa.4.0" = phi i64 [ 1, %top ], [ %47, %cont2 ]
  %29 = load i64, i64* %24, align 8
  %30 = add i64 %29, 1
  %31 = icmp eq i64 %"#temp#1.sroa.4.0", %30
  br i1 %31, label %L33, label %if

if:                                               ; preds = %L3
  %32 = add i64 %"#temp#1.sroa.4.0", -1
  %33 = load i64, i64* %26, align 8
  %34 = icmp ult i64 %32, %33
  br i1 %34, label %idxend, label %oob

L33:                                              ; preds = %L3
  store i8** inttoptr (i64 77348880 to i8**), i8*** %4, align 8
  store i8** inttoptr (i64 77362488 to i8**), i8*** %21, align 8
  store i8** inttoptr (i64 375726352 to i8**), i8*** %20, align 8
  store i8** inttoptr (i64 377039808 to i8**), i8*** %19, align 8
  store i8** %1, i8*** %18, align 8
  %35 = call i8** @"jsys1_#print_to_string#229_39171"(i8** inttoptr (i64 77362448 to i8**), i8*** %4, i32 5)
  store i8** %35, i8*** %10, align 8
  call void @jl_throw(i8** %35)
  unreachable

oob:                                              ; preds = %if
  %36 = alloca i64, align 8
  store i64 %"#temp#1.sroa.4.0", i64* %36, align 8
  call void @jl_bounds_error_ints(i8** %22, i64* nonnull %36, i64 1)
  unreachable

idxend:                                           ; preds = %if
  %37 = load i8***, i8**** %27, align 8
  %38 = getelementptr i8**, i8*** %37, i64 %32
  %39 = load i8**, i8*** %38, align 8
  %40 = icmp eq i8** %39, null
  br i1 %40, label %fail, label %cont2

fail:                                             ; preds = %idxend
  call void @jl_throw(i8** inttoptr (i64 84505768 to i8**))
  unreachable

cont2:                                            ; preds = %idxend
  store i8** %39, i8*** %6, align 8
  %41 = call i8** @jl_gc_pool_alloc(i8* %28, i32 1512, i32 32)
  %42 = getelementptr i8*, i8** %41, i64 -1
  %43 = bitcast i8** %42 to i8***
  store i8** inttoptr (i64 109041296 to i8**), i8*** %43, align 8
  store i8** %41, i8*** %7, align 8
  %44 = getelementptr i8*, i8** %41, i64 1
  %45 = bitcast i8** %44 to i8***
  %46 = bitcast i8** %41 to i64*
  store i64 %"#temp#1.sroa.4.0", i64* %46, align 16
  store i8** %39, i8*** %45, align 8
  %47 = add i64 %"#temp#1.sroa.4.0", 1
  store i8** %39, i8*** %8, align 8
  store i8** %39, i8*** %9, align 8
  %48 = icmp eq i8** %39, %1
  br i1 %48, label %if3, label %L3

if3:                                              ; preds = %cont2
  %sext = shl i64 %"#temp#1.sroa.4.0", 32
  %49 = ashr exact i64 %sext, 32
  %50 = icmp eq i64 %49, %"#temp#1.sroa.4.0"
  br i1 %50, label %pass5, label %fail4

fail4:                                            ; preds = %if3
  call void @jl_throw(i8** inttoptr (i64 77358144 to i8**))
  unreachable

pass5:                                            ; preds = %if3
  %51 = trunc i64 %"#temp#1.sroa.4.0" to i32
  %52 = call i64 inttoptr (i64 1693737504 to i64 (i8**, i32)*)(i8** inttoptr (i64 375726352 to i8**), i32 %51)
  %53 = load i64, i64* %17, align 8
  store i64 %53, i64* %15, align 8
  ret i64 %52
}

I uninstalled Julia 0.5 and 0.52 including folders and the Atom editor. Then I reinstalled Julia 0.6 and in a next step, Atom 1.18.0 x64. Within Atom, the following packages were installed afterwards:

• uber-juno 0.1.2
• julia-client 0.6.0
• ink 0.7.4
• language-julia 0.11.0

From within the Atom editor console I've got after typing Pkg.update() no response,

Then from within julia window: Pkg.update() I've got the following message:

julia> Pkg.update()
INFO: Initializing package repository C:\Users\Carsten\.julia\v0.6

INFO: Cloning METADATA from https://github.com/JuliaLang/METADATA.jl

ERROR: GitError(Code:ERROR, Class:OS, failed to send request: Die Serververbindung konnte nicht hergestellt werden.
)

Stacktrace:
 [1] macro expansion at .\libgit2\error.jl:99 [inlined]

 [2] clone(::String, ::String, ::Base.LibGit2.CloneOptions) at .\libgit2\repository.jl:276

 [3] #clone#100(::String, ::Bool, ::Ptr{Void}, ::Nullable{Base.LibGit2.AbstractCredentials}, ::Function, ::String, ::String) at .\libgit2\libgit2.jl:562

 [4] (::Base.LibGit2.#kw##clone)(::Array{Any,1}, ::Base.LibGit2.#clone, ::String, ::String) at .\:0

 [5] (::Base.Pkg.Dir.##8#10{String,String})() at .\pkg\dir.jl:55

 [6] cd(::Base.Pkg.Dir.##8#10{String,String}, ::String) at .\file.jl:59

 [7] init(::String, ::String) at .\pkg\dir.jl:53

 [8] #cd#1(::Array{Any,1}, ::Function, ::Function, ::String, ::Vararg{Any,N} where N) at .\pkg\dir.jl:28

 [9] update() at .\pkg\pkg.jl:228

In Jupyter notebook, is there any way to re-use the output the line above, within a cell?

Coming from Mathematica, I often find it useful to write things commands which work on the output of the last line using %, here's a stupid example:

Integrate[f[x],x]
Limit[%,x->1] - Limit[%,x->0] 

In general one can write %%% for 3rd-last output etc. https://reference.wolfram.com/language/ref/Out.html

@nostradamus reminds me that underscore _ is the output of the last cell, at least in Python. (Get last answer .) I didn't initially ask this, but would particularly like to be able to do this within a cell, so as to be able to execute multiple steps with one shift-enter.

I would also like to know if there's a way of doing either of these in Julia instead of Python.

My code uses

zeros(Complex{Float64}, data.nuv, nx*nx)

where data.nuv = 1000 and nx = 1000 So, this turns out to be a complex array of 1000 by 1000000. My laptop's RAM is too small; running it on another computer does not have this error.

How can I make an array that is small enough to run without hitting this memory error:

ERROR: OutOfMemoryError()
 in Array{Complex{Float64},N}(::Int64, ::Int64) at ./boot.jl:324
 in zeros(::Type{T}, ::Int64, ::Vararg{Int64,N}) at ./array.jl:169

I am embedding my application inside Julia and I need a good way to read/write the same structures from Julia and C++.

In Python I can simply do:

ffi.cdef("""
    struct keyboard_s {
        int forward;
        int backward;
        int left;
        int right;
        int jump;
    }

    struct keyboard_s *app_get_keyboard();
"""

app = ffi.dlopen("app.dll")

thekeyboard = app.app_get_keyboard();

thekeyboard.forward = 1; # this would immediatly change the memory in C

However, I try something like this in Julia and Julia always just makes copies of the data and isn't able to change the external memory address from C:

type keyboard_s
    forward::Int32
    backward::Int32
    left::Int32
    right::Int32
    jump::Int32
end

# lets imply this would return the memory struct just like app_get_keyboard()
# I just use malloc(sizeof(keyboard_s)) so everybody here can test for themselves...
address = ccall(:malloc, (Int64), (Int64, ), sizeof(keyboard_s))

# address is now a valid Int64 address, so lets map it as pointer of type keyboard_s
ptr = Ptr{keyboard_s}(address)

# thekeyboard contains now the random data from the c static memory
thekeyboard = unsafe_load(ptr)

# this will change only the value of "thekeyboard",
# it doesn't touch C the Int64 address memory pointer...
thekeyboard.forward = 123 # this has no effect on the real memory address :(

# lets load the keyboard again from same address
thekeyboard = unsafe_load(ptr)

thekeyboard.forward == 123 # this is false! no effect whatsoever in C memory from Julia

How am I supposed in Julia to share memory addresses of structs with C?

I was practicing with some exercise like P3.jl. When I run following code, it hasn't any error and the answer is correct.

N = 10
A = rand(N,N)
SA = convert(SharedArray, A)
T = 100
Jacobi_parallel(SA,N,T)

But when I run with N=16386, it has an error:

This case is not supported yet

How can i resolve it?

Thank you.

I have a set S={1,2,3}. I'm trying to create a tuple of the form (i,j,1) for which i and j are elements of S. When S={1,2,3}, my set of tuples (say E)should be {(1,2,1),(2,1,1),(1,3,1),(3,1,1),(3,2,1),(2,3,1)}. I tried in the following way.

    for i in S
      for j in S
        E = Set()
        E = [(i,j,1),(j,i,1), i!=j]
        print(E)
      end
    end

But it doesn't give me the required result.What I'm getting is

Any[(2,2,1),(2,2,1),false]Any[(2,3,1),(3,2,1),true]Any[(2,1,1),(1,2,1),true]Any[(3,2,1),(2,3,1),true]Any[(3,3,1),(3,3,1),false]Any[(3,1,1),(1,3,1),true]Any[(1,2,1),(2,1,1),true]Any[(1,3,1),(3,1,1),true]Any[(1,1,1),(1,1,1),false]

Can someone please help me to get the required result?

Using Emacs 24.5.1 x64 on Windows 7, Julia 0.5.0, julia-mode-20170607.537, and julia-shell-20161125.1110

It appears that julia-mode works fine, as I get syntax highlighting and I can do \delta and it shows a Greek delta.

However running the Julia shell from within Emacs is problematic. After running the instructions and putting the requested several lines in my init.el, I get only a partially running shell.

When I do M-x run-julia, the shell starts up and I can do stuff like 2+2, but there is no julia> prompt, no tab completion, no color highlighting, no help when I type a question mark, and an error when I try \delta .

In fact, when I try \delta , the minibuffer says Search failed: " julia> ", as though the prompt is "in there" somewhere.

When I'm in a Julia buffer and I do , the minibuffer tells me Julia shell is busy! even though nothing is happening in the evidently-crippled shell.

I was also wondering what julia-shell-mode is, which is required-in, since that appears to be the name of the package, but I get the following error in my *messages* buffer: File mode specification error: (file-error "Cannot open load file" "no such file or directory" "julia-shell-mode")

These are the relevant portions of my init.el, copied straight from the instructions.

(defun my-julia-mode-hooks ()
  (require 'julia-shell-mode))
(add-to-list 'load-path "~/.emacs.d/elpa/julia-shell-20161125.1110")
(require 'julia-shell)
(add-hook 'julia-mode-hook 'my-julia-mode-hooks)
(define-key julia-mode-map (kbd "C-c C-c") 'julia-shell-run-region-or-line)
(define-key julia-mode-map (kbd "C-c C-s") 'julia-shell-save-and-go)

Any help is much appreciated, as I've given up an Juno/Atom (too slow, hangs), so this is my next option for an editor with Greek letters and an interactive REPL.

I'm using Julia to solve an integer program. My variables are of the form z[i,j], i in N and j in N and N=10and z[i,j] is a binary variable.

In the first half of the program, I have a set of solutions for which z[1,2]= 1 and z[1,3]=1 and all other variables are zero. Now, I need to pass these values to another set S in such a way that S={1,2,3}. I tried to code it in Julia, but I couldn't get it in the right way. The following is the what I've tried.Here, z_value is the way that I declare my variables z[i,j]. Can someone please help me to make it correct?

for i in N
 for j in N 
   z_value = Pair(i,j)
   if z_value == 1;
     push!(S, Pair(i,j))
     print(S)
   end
 end
end

Edit: Added function header

function backward(l::SoftMax, DLDY::Array{Float64}; kwargs...)
    # credits: https://stats.stackexchange.com/questions/79454/softmax-layer-in-a-neural-network?newreg=d1e89b443dd346ae8bccaf038a944221
    m,n =size(l.x)

    ly = Array{Float64}(n)
    for batch=1:m
      ly = l.y[batch,:]

      for i=1:n
        li = ly[i]
        l.jacobian[:,i] = -li * ly
        l.jacobian[i,i] = li*(1-li)
      end

      # l.jacobian = ly'.*repmat(ly, 1, n)
      # for i=1:n
      #   li = l.y[batch,i]
      #   l.jacobian[i,i] = li*(1.0-li)
      # end

      # # n x 1 = n x n * n x 1
      l.dldx[batch,:] = l.jacobian * DLDY[batch,:]
    end

    return l.dldx

end

Above is my code for the backward function of my softmax layer. The way of computing softmax derivative is described nicely by the answer in this thread https://stats.stackexchange.com/questions/79454/softmax-layer-in-a-neural-network?newreg=d1e89b443dd346ae8bccaf038a944221. Here, I'm looking for a more efficient way to compute the derivative, as the code above takes 0.05~6 seconds to evaluate a 1000 by 100, whereas the previous softmax+cross entropy combined layer takes only 0.002 seconds.

I am thus looking for a way to make the code run faster. I'm not sure if I'm using the most efficient way of computing the jacobian matrix, but I've tried another way where I repmat(ly,1,n) and then dot multiply it with ly. That turns out to be even worse since apparently julia's repmat takes too many allocations.

Essentially, I'm looking for a efficient way to multiply an array with each element in the array, and concat the results into a square matrix. Does any julia master have an idea on this? Thanks!

Is there any way or procedure to implement benders decomposition without constructing the coefficient matrix? For example, I will implement Ax<=b. However, I do not prefer constructing matrix A since A matrix is too large. By the way, is there any function to obtain the coefficient A. I will input the constraints one by one instead of constructing A. So if there is a function like getCoeff(model) which returns the coefficient matrix A, which will be wonderful. All implementations are on Julia. Thanks all.

Given a data file

11616,,,6263,,,1.0
1429,,,5377,,,0.5
1429,,,7207,,,0.5
5377,,,7207,,,0.5
2713,,,1499,,,1.0
8598,,,1031,,,1.0
6166,,,1283,,,0.5
6166,,,12602,,,0.5
...

each entries obey the following structure and comes from the same connected graph {node number1, node number 2, edge weight}(the parsing can be done easily so you may assume it is a 3-tuple data) There is about ~100,000 entries. The matrix is guaranteed to be sparse. How can I convert the these entries into a laplacian matrix and store the laplacian as a output file(or any Julia library I should look at)?

I am trying to make a surface plot without the axis and grid lines. I found that I can turn off the grid with grid = false but I can't find a way to remove the axis lines.

surface(x2d, y2d, z2d, fill_z = color_mat, fc = :haline, grid=false)

Thanks!

I am using Julia version 0.5.2.

julia> addprocs(4);
julia> a=rand(8,8);
julia> distribute(a,2)

When I try to use this codes,I get the error:

ERROR: UndefVarError: distribute not defined

How do i must do?

thanks in advance.

I'm trying to figure out how to create surface plots with Plots.jl. I can create a spherical surface from a 2-d mesh like below:

using Plots
plotlyjs()
# Read the theta and phi angles from file...
x2d = sind(theta2d).*cosd(phi2d)
y2d = sind(theta2d).*sind(phi2d)
z2d = cosd(theta2d)
surface(x2d,y2d,z2d)

However, I want to have the surface color be controlled by a separate matrix like the plot below (made with Python).

from mayavi import mlab
# Create/read plot data...
mlab.figure(bgcolor=(1,1,1), fgcolor=(0.,0.,0.))
mlab.mesh(x2d, y2d, z2d, scalars=p2d, colormap='jet', vmax=5, vmin=-35)

Perhaps I should just use the Python plotting functions directly? Or maybe GLVisualize directly?

Thanks!

I have the following mock setup, with an abstract type, concrete types as subtypes, and a function f which takes two arguments, the first being a Letter.

abstract type Letter end

struct A <: Letter end
struct B <: Letter end

f(::A, x) = ('a', x)
f(::B, x) = ('b', x)

a = A()
b = B()

I would like to define a custom call function for Letter subtypes which simply calls f.

(t::A)(x) = f(t, x)
(t::B)(x) = f(t, x)

While this works, it seems quite redundant, especially considering that there could be many more subtypes of Letter. My attempts are as follows, but neither seem to work.

julia> (t::Letter)(x) = f(t, x)
ERROR: cannot add methods to an abstract type

julia> (t::T)(x) where T <: Letter = f(t, x)
ERROR: function type in method definition is not a type

How can I generalize a call function to match any (concrete) subtype of Letter?