When I compiled Julie with my own userimg.jl sometimes the compile process does not create correct the DLL (System Image Building with help of createDLL.bat). Here's the key command.

%Pathname%\julia.exe "%workDir%build_sysimg_tim.jl" libXJZ core2 "%workDir%userimg.jl" --force

Sometimes, after compilation I can not open the final libXJZ_tim.dll in C/C++ with the help of julia.h (Embedding Julia). I get error in function jl_init_with_image() respectively program only crash .
Here is the print screen
https://drive.google.com/open?id=0ByRuhkKbXjteMjctVk5MQ19qRFE

crashDebug.png1912x935 86.4 KB

I also created mini statistics good/ bad open DLLs (Of course without changing the code, I just called the compilation script).

I created a simple demo in Visual Studio with function foo() to control the DLL opening. Function jl_init_with_image() calls julia_init() where program crash. I modify jl_init_with_image and created own version to detect the problem.

My questions are:
I. How to exit the program correctly when the function julia_init() does not return anything and the program only crash?
(How to capture an exception?,...?)

II. How to ensure stable compilation for system image building to work every created dll?

All the necessary files with good/ bad DLL and Project Visual Studio are in Drive.
This topic is closely related "Static julia compilation with own package"

INFO:
Microsoft Visual Studio Community 2015, Version: 14.0.
On Windows 8.1

julia> versioninfo()
Julia Version 0.5.1
Commit 6445c82 (2017-03-05 13:25 UTC)
Platform Info:
OS: Windows (x86_64-w64-mingw32)
CPU: Intel(R) Xeon(R) CPU E5-2430L v2 @ 2.40GHz
WORD_SIZE: 64
BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Sandybridge)
LAPACK: libopenblas64_
LIBM: libopenlibm
LLVM: libLLVM-3.7.1(ORCJIT, ivybridge)

TimeArray is the type underlying JuliaStats/TimeSeries.
I need a slightly finer distinction and the ability to dispatch over each, RegularTimeArray (where deltaTime is constant) and IrregularTimeArray (where deltaTime is not always the same). I considered using the .meta field to hold Val{:Regular} or Val{:Irregular}. But others use .meta and I do not want to step on that.

What is the most direct, concise and free-of-extra-array-copying way to have each as its own struct and use the machinery of TimeArray for initializing? Can I avoid wrapping TimeArray?

julia> a = [1,2,3,4];
julia> b = a;
julia> a[1] = 4;
julia> b
4-element Array{Int64,1}:
 4
 2
 3
 4

This is the shallow copy AFAIK.

But at a glance, I couldn't find clear differences between copy() and deepcopy(), although Docs says copy() creates a shallow copy of a collection.

Hello

A significant part of my program is to communicate some data back and forth through files, and I would like to maximize the speed of editing a file. Most of the time I will be writing new lines to the end of the file, or writing to the end of a line in the middle of a file. Is there any particular trick to speed up this part of the program?

Thank you.

Try as I might, there is a single computer that I can't seem to get IJulia working on. It's likely a Windows issue, so I'm hoping someone here could spot what it is.

Lots of details in the issue, and I'm ready to post any more diagnostics to get this going.

How do you multi-line statements in this language?
I want to create a macro which will take quite the input and it will be unreadable if the user is forced to put it all in one line.

I'm having issues removing type instabilities from the function below, and would appreciate some tips.

A = rand(36,36);
out = [2]; sysdims = [2,9,2];
@code_warntype ptrace2(A,out,sysdims)

function ptrace2{T<:Number}(A::Matrix{T},out,sysdims)
    nsys = length(sysdims)
    keep = setdiff(1:nsys,out)
    keepdims = sysdims[keep]
    outdims = sysdims[out]::Vector{Int}
    N = prod(keepdims)
    R = Base.product(range.(1,keepdims)...)
    S = Base.product(range.(1,outdims)...)
    B = zeros(T,N,N)
    inds_ii = Vector{Int}(nsys)
    inds_jj = Vector{Int}(nsys)
    for r in R, q in R
        i, j = tindex(q,keepdims), tindex(r,keepdims)
        for l=1:length(keep)
            inds_ii[keep[l]] = q[l]
            inds_jj[keep[l]] = r[l]
        end
        for s in S
            for l=1:length(out)
                inds_ii[out[l]] = s[l]
                inds_jj[out[l]] = s[l]
            end
            ii = tindex(inds_ii,sysdims)
            jj = tindex(inds_jj,sysdims)
            B[i,j] += A[ii,jj]
        end
    end
    return B
end

function tindex(inds,sysdims)
    N = length(inds)
    i = inds[N]
    d = 1
    for n = N-1:-1:1
        d *= sysdims[n+1]
        i += d * (inds[n]-1)
    end
    return i
end

Hello All,
I am working on large machine learning project, where data are stored in S3. My plan is to asynchronously download data to workers that would learn the Neural Network or do other magic on them.

So far, I have encountered two limitations.
Firstly,

AWS.S3.get_bkt(env,bucket_name,options=AWS.S3.GetBucketOptions(max_keys=2000))

returns at most 1000 elements of the bucket, even though I requested more.

Secondly, when I download the file stored as jld, I cannot parse it. After downloading the file issuing

r = AWS.S3.get_object(env,bkt,file_name);

r.obj is of type Array{UInt8,1} but I cannot load it using load(IOBuffer(r.obj)).

Has anyone tried to overcome these issues?

Thanks for suggestions.
Tomas

In CPLEX we have option to detect conflicting equations and/ check infeasibility of the model,if any. I am using Clp solver. Is there any option to do aforementioned task?

Here is example Julia code that reverses element of touple that I'm trying to understand. It must be my lack of experience but it's extremely confusing to me why this is working at all.

revargs() = ()
revargs(x, r...) = (revargs(r...)...,x)

reverse(t::Tuple) = revargs(t...)

julia> data = (1,2,3,4)
(1,2,3,4)

julia> reverse(data)
(4,3,2,1)

I'm trying to understand two things:
- how ... operator works here to produce reversed result.
- why it it necessary to define revargs() = () even though it does not look like it is being called to produce the result of reverse(data)

Can you please explain to me what happens step by step after I call reverse(data)?

I'm sorry if my question is to broad or unclear but i'm really stuck and don't know how to find the answer.

Hi.

Julia tells me that "type definition not allowed inside a local scope".

I have a simple question - why?

With Python 3 as a backend, what's the solution to this error?

julia> writer("a")
ERROR: PyError (ccall(@pysym(:PyObject_Call), PyPtr, (PyPtr, PyPtr, PyPtr), o, arg, C_NULL)) 
TypeError('expected bytes, str found',)

I also tried

julia> writer(Vector{UInt8}("a"))
ERROR: PyError (ccall(@pysym(:PyObject_Call), PyPtr, (PyPtr, PyPtr, PyPtr), o, arg, C_NULL)) 
TypeError('expected bytes, bytearray found',)

1. How to choose 0.6 for appveyor testing? nightly now points to 0.7
2. Can coveralls.io be configured to display the highest coverage within a set of build?

I am posting for a friend who tried installing JuliaPro in mainland China. He said the installation got to the stage of copying files and the progress froze, getting nowhere for 2 hours before he shut it down.

His network speed should be pretty good. The only guess we have: maybe JuliaPro needs to download some packages from a US server which is blocked by the famous (or infamous ) Chinese national firewall?

He had installed Python and R on the same machine before without any problem, but all the packages they needed could be located on mirror servers in China. Are there mirror servers for Julia packages in China?

And is anybody experiencing same problem (or no problem at all) installing JuliaPro from mainland China?

Hi,

Are there any plans on updating Cassandra drivers for Julia?
I've seen some that has been written for v.0.4 that failed the tests. It would be really awesome to have!

Appveyor gets the following error from Compat 0.25.2 on Julia 0.5.2:

ERROR: LoadError: LoadError: syntax: unhandled expr (error #) in include_from_node1(::String) at .\loading.jl:488 (repeats 2 times) in macro expansion; at .\none:2 [inlined] in anonymous at .\:? in eval(::Module, ::Any) at .\boot.jl:234 in process_options(::Base.JLOptions) at .\client.jl:242 in _start() at .\client.jl:321 while loading C:\Users\appveyor\.julia\v0.5\MXNet\src\base.jl, in expression starting on line 1 while loading C:\Users\appveyor\.julia\v0.5\MXNet\src\MXNet.jl, in expression starting on line 24 ERROR: LoadError: Failed to precompile MXNet to C:\Users\appveyor\.julia\lib\v0.5\MXNet.ji. in compilecache(::String) at .\loading.jl:593 in require(::Symbol) at .\loading.jl:422 in include_from_node1(::String) at .\loading.jl:488 in process_options(::Base.JLOptions) at .\client.jl:265 in _start() at .\client.jl:321 while loading C:\Users\appveyor\.julia\v0.5\MXNet\test\runtests.jl, in expression starting on line 1

When I test locally on 0.6 I get no errors. I suppose this is not too surprising since Compat probably sees that it is on 0.6 and doesn't get called. I guess my question is, is it even possible to use @compat on struct, and, if so, what's causing this error?

As a beginner programmer who's relatively new to Julia, I'm a bit confused about how method definitions work. If I define a simple function as follows:

function xy{T<:Real}(x::Vector{T}, y::Vector{T})
    out = x .* y
end

Why do I get a MethodError if, for example, x is of type Int64 and y is of type Float64? There's a similar example in the Methods documentation, but it's still not clear to me why both types can't be used in the function if they are both subtypes of Real. This may stem from my misunderstanding of how type parameters are used in the method signature.

I often work with data that can be of various types and it would be rather cumbersome to convert them all to similar types prior to executing a function. For example, the function above wouldn't work if x was of type Float32 and y was of type Float64. A trivial solution would be a more generalized form of the function:

function xy(x::Vector, y::Vector)
    out = x .* y
end

However, I feel that this isn't ideal. If I wanted to add a two-dimensional array to the parameter tuple, I'm not sure how I would explicitly enforce the dimensionality of the array without the presence of a method type parameter T. For example, a function with definition as follows:

function xyz{T<:Real}(x::Vector{T}, y::Vector{T}, z::Array{T,2})

would require z to be a 2-dimensional array, but also require it have the same type as x and y.

I was hoping someone could provide me with some clarity on the technical aspects involved with this. And more importantly, what is the recommended "Julian" way of proceeding if x, y, and z are of different types?

Please let me know the way to change variable name and constraint name. We usually do it in CPLEX to change default name of variables and constraints to make it more meaningful and to give better understanding of .lp file.

Hi all,

I have a function using data which has to be initialized at run-time. Here is a basic example:

type Test{T<:Number}
       x::T
       a::Array{T, 1}
end
function f(t::Test, x)
       i = floor(Integer, x - t.x)
       return x*t.a[i]
end
(t::Test)(x) = f(t, x) #functor definition
t = Test(3., rand(100)); #initialize data

if I map this function on an array, I get quite different performance whether I use the function f or the functor-call:

#init
rsi	= rand(10)*100 + 4;
@timed map(x -> f(t, x), rsi);
map(t, rsi);			
#test	
rs	= rand(10^7)*100 + 4;	
@time map(x -> f(t, x), rs);  
#1.349930 seconds (20.03 M allocations: 382.873 MB, 26.17% gc time)
@time map(t, rs);
#  0.290798 seconds (7 allocations: 76.294 MB, 0.83% gc time)

The difference is even bigger when I use vectorized functions:

#init
ft(x) = f(t, x)
ft.(rsi);
t.(rsi);
#test
@time ft.(rs);
#2.698942 seconds (30.00 M allocations: 534.059 MB, 8.62% gc time)
@time t.(rs);
#0.305188 seconds (7 allocations: 76.294 MB, 1.57% gc time)

So far so good, if I want to map the function I simply have to use the functor. However, what am I going to do if I have additional functions which need the same initialized data? I could of course define a type for each function, but that does not seem very elegant.

Any suggestions? Am I missing something?

I've been trying for a few days to add the following constraint in jump:

@NLconstraint(mod, R[i,j,h]^2 + T[i,j,h]^2 - 2*u_ijk[i,i,j,h]*u_ijk[j,j,i,h]<=0)

I tried every academic solver, but none is able to recognize that it is a rotated cone constraint.
I found ways to declare second order cone constraints (not rotated) that should work with pajarito and Gurobi, yet I didn't manage to put my constraint in the required form.

Is there a solver that supports this kind of constraints or a standard procedure to reformulate it? having more than 100 of those constraints the difference between a NLconstraint and a conic one is huge (and atm nlsolvers are giving me suboptimal solutions).

Thanks for the help

I tried to compile Julie with my own package "FOO" with the help of script. I ran it with the following command line arguments.

julia.exe "C:/Julia-0.5.0/build_sysimg.jl" libBar core2 "C:/Julia-0.5.0/userimg.jl" --force

My userimg.jl looked like

using FOO

@Base.ccallable Float64 function jl_foo_fcn(vec1::Vector{Float64}, vec2::Vector{Float64})

	return FOO.mape(vec1, vec2)
end

@Base.ccallable Int64 function jl_foo_model(cestaT::String, cestaR::String, pathC::String)

  FOO.model(cestaT, cestaR, pathC)
  return 0
end

After compilation I can not open the final libBar.dll in C with the help of julia.h. I get error in function jl_init_with_image()

Unhandled exception at 0x0000000064F3C4BA (libjulia.dll) in TIM.exe: 0xC00000FD: Stack overflow (parameters: 0x0000000000000001, 0x000000FEC0D33FF8).

How correct setup commands for compilation?
If I ran Julia from folder "JULIA_HOME/bin" with command

julia.exe -J libBar.dll --compile=no --precompiled=yes

it works with functions from userimg.jl.
But, how I should compile DLL which contains my package and was executable separately?

Thanks in advance,
Martin

YAML.jl has recently gained support for custom tag constructors (YAML.jl/pull/33) and to understand how they work, I thought I'd try it out to implement an !include tag that can be used to include/merge the contents of arbitrary files into YAML files at parse time.

e.g. if you have files foo.yaml and bar.yaml

# foo.yaml
x: !include bar.yaml
y: [1, 2, 3]

# bar.yaml
a: 1
b: 2

through some process, these should effectively become:

x:
  a: 1
  b: 2
y: [1, 2, 3]

Ideally, this would work for relative and absolute paths

Here is a solution using the new tag constructor features in YAML.jl:

module yaml

using YAML

const _constructors = Dict{AbstractString, Function}()

"""Loads YAML files relative to the current directory"""
function load_file_relative(filename::AbstractString;
    constructors::Dict{AbstractString, Function}=_constructors
)
    filename = abspath(filename)
    cd(dirname(filename)) do
        YAML.load_file(filename, constructors)
    end
end

"""Processes an !include constructor"""
function construct_include(constructor::YAML.Constructor, node::YAML.ScalarNode)
    filename = node.value
    if !isfile(filename)
        throw(YAML.ConstructorError(nothing, nothing, "file $(abspath(filename)) does not exist", node.start_mark))
    end
    if !(last(splitext(filename)) in (".yaml", ".yml", ".json"))
        return readstring(filename)
    end
    # pass forward custom constructors
    constructors = Dict{AbstractString, Function}(
        tag => f for (tag, f) in constructor.yaml_constructors
        if tag !== nothing && ismatch(r"^!\w+$", tag)
    )
    load_file_relative(filename; constructors=constructors)
end
construct_include(::YAML.Constructor, node::YAML.Node) = throw(
    YAML.ConstructorError(nothing, nothing, "expected a scalar node, but found $(typeof(node))", node.start_mark)
)
_constructors["!include"] = construct_include

end

Then, foo.yaml can be loaded as follows:

julia> yaml.load_file_relative("/tmp/foo.yaml")
Dict{Any,Any} with 2 entries:
  "x" => Dict{Any,Any}(Pair{Any,Any}("b",2),Pair{Any,Any}("a",1))
  "y" => [1,2,3]

Hopefully this is useful to someone!

The following is part of the source code of marching_tetrahedra.jl in the Meshing.jl module.

The tetEdgeCrnrs and tetTri fields of the VoxelIndices immutable don't really make sense to me, so I would appreciate the help. Thank you.

"""
Voxel corner and edge indexing conventions

        Z
        |

        5------5------6              Extra edges not drawn
       /|            /|              -----------
      8 |           6 |              - face diagonals
     /  9          /  10                - 13: 1 to 3
    8------7------7   |                 - 14: 1 to 8
    |   |         |   |                 - 15: 1 to 6
    |   1------1--|---2  -- Y           - 16: 5 to 7
    12 /          11 /                  - 17: 2 to 7
    | 4           | 2                   - 18: 4 to 7
    |/            |/                 - body diagonal
    4------3------3                     - 19: 1 to 7

  /
 X
"""
immutable VoxelIndices{T <: Integer}
    voxCrnrPos::NTuple{8,NTuple{3,T}}
    voxEdgeCrnrs::NTuple{19, NTuple{2,T}}
    voxEdgeDir::NTuple{19,T}
    voxEdgeIx::NTuple{8,NTuple{8,T}}
    subTets::NTuple{6,NTuple{4,T}}
    tetEdgeCrnrs::NTuple{6,NTuple{2,T}}
    tetTri::NTuple{16,NTuple{6,T}}

    function VoxelIndices()
        voxCrnrPos = ((0, 0, 0),
                      (0, 1, 0),
                      (1, 1, 0),
                      (1, 0, 0),
                      (0, 0, 1),
                      (0, 1, 1),
                      (1, 1, 1),
                      (1, 0, 1))
        # the voxel IDs at either end of the tetrahedra edges, by edge ID
        voxEdgeCrnrs = ((1, 2),
                        (2, 3),
                        (4, 3),
                        (1, 4),
                        (5, 6),
                        (6, 7),
                        (8, 7),
                        (5, 8),
                        (1, 5),
                        (2, 6),
                        (3, 7),
                        (4, 8),
                        (1, 3),
                        (1, 8),
                        (1, 6),
                        (5, 7),
                        (2, 7),
                        (4, 7),
                        (1, 7))

        # direction codes:
        # 0 => +x, 1 => +y, 2 => +z,
        # 3 => +xy, 4 => +xz, 5 => +yz, 6 => +xyz
        voxEdgeDir = (1,0,1,0,1,0,1,0,2,2,2,2,3,4,5,3,4,5,6)

        # For a pair of corner IDs, the edge ID joining them
        # 0 denotes a pair with no edge
        voxEdgeIx = ((0,1,13,4,9,15,19,14),
                     (1,0,2,0,0,10,17,0),
                     (13,2,0,3,0,0,11,0),
                     (4,0,3,0,0,0,18,12),
                     (9,0,0,0,0,5,16,8),
                     (15,10,0,0,5,0,6,0),
                     (19,17,11,18,16,6,0,7),
                     (14,0,0,12,8,0,7,0))

        # voxel corners that comprise each of the six tetrahedra
        subTets = ((1,3,2,7),
                      (1,8,4,7),
                      (1,4,3,7),
                      (1,2,6,7),
                      (1,5,8,7),
                      (1,6,5,7))
        # tetrahedron corners for each edge (indices 1-4)
        tetEdgeCrnrs = ((1,2),
                        (2,3),
                        (1,3),
                        (1,4),
                        (2,4),
                        (3,4))

        # triangle cases for a given tetrahedron edge code
        tetTri = ((0,0,0,0,0,0),
                    (1,3,4,0,0,0),
                    (1,5,2,0,0,0),
                    (3,5,2,3,4,5),
                    (2,6,3,0,0,0),
                    (1,6,4,1,2,6),
                    (1,5,6,1,6,3),
                    (4,5,6,0,0,0),
                    (4,6,5,0,0,0),
                    (1,6,5,1,3,6),
                    (1,4,6,1,6,2),
                    (2,3,6,0,0,0),
                    (3,2,5,3,5,4),
                    (1,2,5,0,0,0),
                    (1,4,3,0,0,0),
                    (0,0,0,0,0,0))

        new(voxCrnrPos,
            voxEdgeCrnrs,
            voxEdgeDir,
            voxEdgeIx,
            subTets,
            tetEdgeCrnrs,
            tetTri)
    end
end

I am trying to figure out the mesh importing and exporting capabilities of Julia. I came across VTK.jl but but it was hardly documented, so I wonder, is it possible to use VTK.jl to load and save vtk or pvd file formats?

Thank you.

Say I opened a video like this:

f = "path2video.file"
cmd = is_windows() ? `start $f` : is_linux() ? `xdg-open $f` : is_apple() ? `open $f` : error("Unknown OS")
run(cmd)

How can I kill this process from within Julia later on?

What is a recommended way to handle exception/errors in larger projects?
Is there similar mechanism as RAII in C++ (can I be sure that a resource is cleaned if an exception is thrown; or have I always use finally and handle resource)?

Assume an exception with some additional info, see C++ example below, what is a best way in Julia...

class MyException : public std::exception
{
public:
  explicit MyException(const char* msg) :MyException(0, msg) {}
  explicit MyException(int code, const char* msg) : _code{ code }, std::exception{ msg } {}
  int code() { return _code; }

protected:
  int _code;
};

Could someone please explain how the compat macro works from Compat.jl? I went through the source code, but it wasn't very clear. It seems to enable defining constructors for families of types simultaneously, is that right, and is it the only way it can be used? Thanks a lot in advance.

I am trying to visualize a 3D Vectorfield like this:

image.png946x367 355 KB

Is there any Julia library capable of creating plots like these?

Hi ,
What is the best way to set up cluster using local lab PCs for parallel Julia computations?
In our lab we have hybrid environment (Windows-Linux) dominated by Windows servers.
I have seen ClusterManagers and MPI libraries, but ClusterManagers jl seems to support Linux environment and MPImanager is said to support Julia 0.4 only.
Could you please give me some advice for this use-case?

Hi,

I am developing a package that, among other things, runs user-defined functions that may allocate data, open an write to files and run external programs. Is there a function similar to workspace() that I can call inside a script to make sure all resources allocated by a function are freed?

In more detail, I need to cleanly rerun user-defined functions multiple times and I would like to know if there is a way to force or make sure that, regarding those function calls, I/O is done, opened files are closed, allocated variables are freed and external programs finished.

Thank you!