I/O and Network¶
General I/O¶
STDOUT
¶Global variable referring to the standard out stream.
STDERR
¶Global variable referring to the standard error stream.
STDIN
¶Global variable referring to the standard input stream.
open
(filename[, read, write, create, truncate, append]) → IOStream¶Open a file in a mode specified by five boolean arguments. The default is to open files for reading only. Returns a stream for accessing the file.
open
(filename[, mode]) → IOStreamAlternate syntax for open, where a string-based mode specifier is used instead of the five booleans. The values of
mode
correspond to those fromfopen(3)
or Perlopen
, and are equivalent to setting the following boolean groups:Mode Description r read r+ read, write w write, create, truncate w+ read, write, create, truncate a write, create, append a+ read, write, create, append
open
(command, mode::AbstractString="r", stdio=DevNull)Start running
command
asynchronously, and return a tuple(stream,process)
. Ifmode
is"r"
, thenstream
reads from the process’s standard output andstdio
optionally specifies the process’s standard input stream. Ifmode
is"w"
, thenstream
writes to the process’s standard input andstdio
optionally specifies the process’s standard output stream.
open
(f::Function, command, mode::AbstractString="r", stdio=DevNull)Similar to
open(command,mode,stdio)
, but callsf(stream)
on the resulting read or write stream, then closes the stream and waits for the process to complete. Returns the value returned byf
.
open
(f::Function, args...)Apply the function
f
to the result ofopen(args...)
and close the resulting file descriptor upon completion.Example:
open(readstring,"file.txt")
IOBuffer
() → IOBuffer¶Create an in-memory I/O stream.
IOBuffer
(size::Int)Create a fixed size IOBuffer. The buffer will not grow dynamically.
IOBuffer
(string)Create a read-only IOBuffer on the data underlying the given string.
IOBuffer
([data][, readable, writable[, maxsize]])Create an IOBuffer, which may optionally operate on a pre-existing array. If the readable/writable arguments are given, they restrict whether or not the buffer may be read from or written to respectively. By default the buffer is readable but not writable. The last argument optionally specifies a size beyond which the buffer may not be grown.
takebuf_array
(b::IOBuffer)¶Obtain the contents of an
IOBuffer
as an array, without copying. Afterwards, theIOBuffer
is reset to its initial state.
takebuf_string
(b::IOBuffer)¶Obtain the contents of an
IOBuffer
as a string, without copying. Afterwards, the IOBuffer is reset to its initial state.
fdio
([name::AbstractString, ]fd::Integer[, own::Bool]) → IOStream¶Create an
IOStream
object from an integer file descriptor. Ifown
istrue
, closing this object will close the underlying descriptor. By default, anIOStream
is closed when it is garbage collected.name
allows you to associate the descriptor with a named file.
flush
(stream)¶Commit all currently buffered writes to the given stream.
close
(stream)¶Close an I/O stream. Performs a
flush
first.
write
(stream::IO, x)¶write
(filename::AbstractString, x)Write the canonical binary representation of a value to the given I/O stream or file. Returns the number of bytes written into the stream.
You can write multiple values with the same
write
call. i.e. the following are equivalent:write(stream,x,y...)write(stream,x)+write(stream,y...)
read
(stream::IO, T)¶Read a single value of type
T
fromstream
, in canonical binary representation.
read
(stream::IO, T, dims)Read a series of values of type
T
fromstream
, in canonical binary representation.dims
is either a tuple or a series of integer arguments specifying the size of theArray{T}
to return.
read!
(stream::IO, array::Union{Array, BitArray})¶read!
(filename::AbstractString, array::Union{Array, BitArray})Read binary data from an I/O stream or file, filling in
array
.
readbytes!
(stream::IO, b::AbstractVector{UInt8}, nb=length(b); all=true)¶Read at most
nb
bytes fromstream
intob
, returning the number of bytes read. The size ofb
will be increased if needed (i.e. ifnb
is greater thanlength(b)
and enough bytes could be read), but it will never be decreased.See
read
for a description of theall
option.
read
(s::IO, nb=typemax(Int))Read at most
nb
bytes froms
, returning aVector{UInt8}
of the bytes read.
read
(s::IOStream, nb::Integer; all=true)Read at most
nb
bytes froms
, returning aVector{UInt8}
of the bytes read.If
all
istrue
(the default), this function will block repeatedly trying to read all requested bytes, until an error or end-of-file occurs. Ifall
isfalse
, at most oneread
call is performed, and the amount of data returned is device-dependent. Note that not all stream types support theall
option.
read
(filename::AbstractString, args...)Open a file and read its contents.
args
is passed toread
: this is equivalent toopen(io->read(io,args...),filename)
.
unsafe_read
(io, ref, nbytes)¶Copy nbytes from the IO stream object into ref (converted to a pointer).
It is recommended that subtypes
T<:IO
override the following method signature to provide more efficient implementations:unsafe_read(s::T,p::Ptr{UInt8},n::UInt)
unsafe_write
(io, ref, nbytes)¶Copy nbytes from ref (converted to a pointer) into the IO stream object.
It is recommended that subtypes
T<:IO
override the following method signature to provide more efficient implementations:unsafe_write(s::T,p::Ptr{UInt8},n::UInt)
position
(s)¶Get the current position of a stream.
seek
(s, pos)¶Seek a stream to the given position.
seekstart
(s)¶Seek a stream to its beginning.
seekend
(s)¶Seek a stream to its end.
skip
(s, offset)¶Seek a stream relative to the current position.
mark
(s)¶Add a mark at the current position of stream
s
. Returns the marked position.See also
unmark()
,reset()
,ismarked()
.
unmark
(s)¶Remove a mark from stream
s
. Returnstrue
if the stream was marked,false
otherwise.See also
mark()
,reset()
,ismarked()
.
reset
(s)¶Reset a stream
s
to a previously marked position, and remove the mark. Returns the previously marked position. Throws an error if the stream is not marked.See also
mark()
,unmark()
,ismarked()
.
eof
(stream) → Bool¶Tests whether an I/O stream is at end-of-file. If the stream is not yet exhausted, this function will block to wait for more data if necessary, and then return
false
. Therefore it is always safe to read one byte after seeingeof
returnfalse
.eof
will returnfalse
as long as buffered data is still available, even if the remote end of a connection is closed.
isreadonly
(stream) → Bool¶Determine whether a stream is read-only.
iswritable
(io) → Bool¶Returns
true
if the specified IO object is writable (if that can be determined).
isreadable
(io) → Bool¶Returns
true
if the specified IO object is readable (if that can be determined).
isopen
(object) → Bool¶Determine whether an object - such as a stream, timer, or mmap – is not yet closed. Once an object is closed, it will never produce a new event. However, a closed stream may still have data to read in its buffer, use
eof
to check for the ability to read data. Usepoll_fd
to be notified when a stream might be writable or readable.
serialize
(stream, value)¶Write an arbitrary value to a stream in an opaque format, such that it can be read back by
deserialize
. The read-back value will be as identical as possible to the original. In general, this process will not work if the reading and writing are done by different versions of Julia, or an instance of Julia with a different system image.Ptr
values are serialized as all-zero bit patterns (NULL
).
deserialize
(stream)¶Read a value written by
serialize
.deserialize
assumes the binary data read fromstream
is correct and has been serialized by a compatible implementation ofserialize
. It has been designed with simplicity and performance as a goal and does not validate the data read. Malformed data can result in process termination. The caller has to ensure the integrity and correctness of data read fromstream
.
escape_string
(io, str::AbstractString, esc::AbstractString)¶General escaping of traditional C and Unicode escape sequences, plus any characters in esc are also escaped (with a backslash).
unescape_string
(io, s::AbstractString)¶General unescaping of traditional C and Unicode escape sequences. Reverse of
escape_string()
.
join
(io, items, delim[, last])¶Print elements of
items
toio
withdelim
between them. Iflast
is specified, it is used as the final delimiter instead ofdelim
.
print_shortest
(io, x)¶Print the shortest possible representation, with the minimum number of consecutive non-zero digits, of number
x
, ensuring that it would parse to the exact same number.
fd
(stream)¶Returns the file descriptor backing the stream or file. Note that this function only applies to synchronous
File
‘s andIOStream
‘s not to any of the asynchronous streams.
redirect_stdout
()¶Create a pipe to which all C and Julia level
STDOUT
output will be redirected. Returns a tuple(rd,wr)
representing the pipe ends. Data written toSTDOUT
may now be read from the rd end of the pipe. The wr end is given for convenience in case the oldSTDOUT
object was cached by the user and needs to be replaced elsewhere.
redirect_stdout
(stream)Replace
STDOUT
by stream for all C and Julia level output toSTDOUT
. Note thatstream
must be a TTY, aPipe
or aTCPSocket
.
redirect_stderr
([stream])¶Like
redirect_stdout
, but forSTDERR
.
redirect_stdin
([stream])¶Like redirect_stdout, but for STDIN. Note that the order of the return tuple is still (rd,wr), i.e. data to be read from STDIN, may be written to wr.
readchomp
(x)¶Read the entirety of
x
as a string and remove a single trailing newline. Equivalent tochomp(readstring(x))
.
truncate
(file, n)¶Resize the file or buffer given by the first argument to exactly
n
bytes, filling previously unallocated space with ‘\0’ if the file or buffer is grown.
skipchars
(stream, predicate; linecomment::Char)¶Advance the stream until before the first character for which
predicate
returnsfalse
. For exampleskipchars(stream,isspace)
will skip all whitespace. If keyword argumentlinecomment
is specified, characters from that character through the end of a line will also be skipped.
countlines
(io[, eol::Char])¶Read
io
until the end of the stream/file and count the number of lines. To specify a file pass the filename as the first argument. EOL markers other than ‘\n’ are supported by passing them as the second argument.
PipeBuffer
()¶An IOBuffer that allows reading and performs writes by appending. Seeking and truncating are not supported. See IOBuffer for the available constructors.
PipeBuffer
(data::Vector{UInt8}[, maxsize])Create a PipeBuffer to operate on a data vector, optionally specifying a size beyond which the underlying Array may not be grown.
readavailable
(stream)¶Read all available data on the stream, blocking the task only if no data is available. The result is a
Vector{UInt8,1}
.
IOContext
¶IOContext provides a mechanism for passing output configuration settings among
show
methods.In short, it is an immutable dictionary that is a subclass of
IO
. It supports standard dictionary operations such asgetindex
, and can also be used as an I/O stream.
IOContext
(io::IO, KV::Pair)Create an
IOContext
that wraps a given stream, adding the specifiedkey=>value
pair to the properties of that stream (note thatio
can itself be anIOContext
).- use
(key=>value)indict
to see if this particular combination is in the properties set - use
get(dict,key,default)
to retrieve the most recent value for a particular key
The following properties are in common use:
:compact
: Boolean specifying that small values should be printed more compactly, e.g. that numbers should be printed with fewer digits. This is set when printing array elements.:limit
: Boolean specifying that containers should be truncated, e.g. showing…
in place of most elements.:displaysize
: ATuple{Int,Int}
giving the size in rows and columns to use for text output. This can be used to override the display size for called functions, but to get the size of the screen use thedisplaysize
function.
- use
IOContext
(io::IO, context::IOContext)Create an
IOContext
that wraps an alternateIO
but inherits the properties ofcontext
.
IOContext
(io::IO; properties...)The same as
IOContext(io::IO,KV::Pair)
, but accepting properties as keyword arguments.
Text I/O¶
show
(x)¶Write an informative text representation of a value to the current output stream. New types should overload
show(io,x)
where the first argument is a stream. The representation used byshow
generally includes Julia-specific formatting and type information.
showcompact
(x)¶Show a compact representation of a value.
This is used for printing array elements without repeating type information (which would be redundant with that printed once for the whole array), and without line breaks inside the representation of an element.
To offer a compact representation different from its standard one, a custom type should test
get(io,:compact,false)
in its normalshow
method.
showall
(x)¶Similar to
show
, except shows all elements of arrays.
summary
(x)¶Return a string giving a brief description of a value. By default returns
string(typeof(x))
, e.g.Int64
.For arrays, returns a string of size and type info, e.g.
10-elementArray{Int64,1}
.
print
(x)¶Write (to the default output stream) a canonical (un-decorated) text representation of a value if there is one, otherwise call
show
. The representation used byprint
includes minimal formatting and tries to avoid Julia-specific details.
print_with_color
(color::Symbol, [io, ]strings...)¶Print strings in a color specified as a symbol.
color
may take any of the values:normal
,:bold
,:black
,:blue
,:cyan
,:green
,:magenta
,:red
,:white
, or:yellow
.
info
(msg)¶Display an informational message. Argument
msg
is a string describing the information to be displayed.
warn
(msg)¶Display a warning. Argument
msg
is a string describing the warning to be displayed.
@printf
([io::IOStream, ]"%Fmt", args...)¶Print
args
using Cprintf()
style format specification string. Optionally, anIOStream
may be passed as the first argument to redirect output.
@sprintf
("%Fmt", args...)¶Return
@printf
formatted output as string.julia>s=@sprintf"this is a %s%15.1f""test"34.567;julia>println(s)thisisatest34.6
sprint
(f::Function, args...)¶Call the given function with an I/O stream and the supplied extra arguments. Everything written to this I/O stream is returned as a string.
showerror
(io, e)¶Show a descriptive representation of an exception object.
dump
(x)¶Show every part of the representation of a value.
readstring
(stream::IO)¶readstring
(filename::AbstractString)Read the entire contents of an I/O stream or a file as a string. The text is assumed to be encoded in UTF-8.
readline
(stream::IO=STDIN)¶readline
(filename::AbstractString)Read a single line of text, including a trailing newline character (if one is reached before the end of the input), from the given I/O stream or file (defaults to
STDIN
). When reading from a file, the text is assumed to be encoded in UTF-8.
readuntil
(stream::IO, delim)¶readuntil
(filename::AbstractString, delim)Read a string from an I/O stream or a file, up to and including the given delimiter byte. The text is assumed to be encoded in UTF-8.
readlines
(stream::IO)¶readlines
(filename::AbstractString)Read all lines of an I/O stream or a file as a vector of strings. The text is assumed to be encoded in UTF-8.
eachline
(stream::IO)¶eachline
(filename::AbstractString)Create an iterable object that will yield each line from an I/O stream or a file. The text is assumed to be encoded in UTF-8.
readdlm
(source, delim::Char, T::Type, eol::Char; header=false, skipstart=0, skipblanks=true, use_mmap, quotes=true, dims, comments=true, comment_char='#')¶Read a matrix from the source where each line (separated by
eol
) gives one row, with elements separated by the given delimiter. The source can be a text file, stream or byte array. Memory mapped files can be used by passing the byte array representation of the mapped segment as source.If
T
is a numeric type, the result is an array of that type, with any non-numeric elements asNaN
for floating-point types, or zero. Other useful values ofT
includeString
,AbstractString
, andAny
.If
header
istrue
, the first row of data will be read as header and the tuple(data_cells,header_cells)
is returned instead of onlydata_cells
.Specifying
skipstart
will ignore the corresponding number of initial lines from the input.If
skipblanks
istrue
, blank lines in the input will be ignored.If
use_mmap
istrue
, the file specified bysource
is memory mapped for potential speedups. Default istrue
except on Windows. On Windows, you may want to specifytrue
if the file is large, and is only read once and not written to.If
quotes
istrue
, columns enclosed within double-quote (”) characters are allowed to contain new lines and column delimiters. Double-quote characters within a quoted field must be escaped with another double-quote. Specifyingdims
as a tuple of the expected rows and columns (including header, if any) may speed up reading of large files. Ifcomments
istrue
, lines beginning withcomment_char
and text followingcomment_char
in any line are ignored.
readdlm
(source, delim::Char, eol::Char; options...)If all data is numeric, the result will be a numeric array. If some elements cannot be parsed as numbers, a heterogeneous array of numbers and strings is returned.
readdlm
(source, delim::Char, T::Type; options...)The end of line delimiter is taken as
\n
.
readdlm
(source, delim::Char; options...)The end of line delimiter is taken as
\n
. If all data is numeric, the result will be a numeric array. If some elements cannot be parsed as numbers, a heterogeneous array of numbers and strings is returned.
readdlm
(source, T::Type; options...)The columns are assumed to be separated by one or more whitespaces. The end of line delimiter is taken as
\n
.
readdlm
(source; options...)The columns are assumed to be separated by one or more whitespaces. The end of line delimiter is taken as
\n
. If all data is numeric, the result will be a numeric array. If some elements cannot be parsed as numbers, a heterogeneous array of numbers and strings is returned.
writedlm
(f, A, delim='\t'; opts)¶Write
A
(a vector, matrix, or an iterable collection of iterable rows) as text tof
(either a filename string or anIO
stream) using the given delimiterdelim
(which defaults to tab, but can be any printable Julia object, typically aChar
orAbstractString
).For example, two vectors
x
andy
of the same length can be written as two columns of tab-delimited text tof
by eitherwritedlm(f,[xy])
or bywritedlm(f,zip(x,y))
.
readcsv
(source, [T::Type]; options...)¶Equivalent to
readdlm
withdelim
set to comma, and type optionally defined byT
.
writecsv
(filename, A; opts)¶Equivalent to
writedlm()
withdelim
set to comma.
Base64EncodePipe
(ostream)¶Returns a new write-only I/O stream, which converts any bytes written to it into base64-encoded ASCII bytes written to
ostream
. Callingclose
on theBase64EncodePipe
stream is necessary to complete the encoding (but does not closeostream
).
Base64DecodePipe
(istream)¶Returns a new read-only I/O stream, which decodes base64-encoded data read from
istream
.
base64encode
(writefunc, args...)¶base64encode
(args...)Given a
write
-like functionwritefunc
, which takes an I/O stream as its first argument,base64encode(writefunc,args...)
callswritefunc
to writeargs...
to a base64-encoded string, and returns the string.base64encode(args...)
is equivalent tobase64encode(write,args...)
: it converts its arguments into bytes using the standardwrite
functions and returns the base64-encoded string.
base64decode
(string)¶Decodes the base64-encoded
string
and returns aVector{UInt8}
of the decoded bytes.
displaysize
(io) → (lines, columns)¶Return the nominal size of the screen that may be used for rendering output to this io object
Multimedia I/O¶
Just as text output is performed by print
and user-defined types
can indicate their textual representation by overloading show
,
Julia provides a standardized mechanism for rich multimedia output
(such as images, formatted text, or even audio and video), consisting
of three parts:
- A function
display(x)
to request the richest available multimedia display of a Julia objectx
(with a plain-text fallback). - Overloading
show
allows one to indicate arbitrary multimedia representations (keyed by standard MIME types) of user-defined types. - Multimedia-capable display backends may be registered by subclassing
a generic
Display
type and pushing them onto a stack of display backends viapushdisplay
.
The base Julia runtime provides only plain-text display, but richer displays may be enabled by loading external modules or by using graphical Julia environments (such as the IPython-based IJulia notebook).
display
(x)¶display
(d::Display, x)display
(mime, x)display
(d::Display, mime, x)Display
x
using the topmost applicable display in the display stack, typically using the richest supported multimedia output forx
, with plain-textSTDOUT
output as a fallback. Thedisplay(d,x)
variant attempts to displayx
on the given displayd
only, throwing aMethodError
ifd
cannot display objects of this type.There are also two variants with a
mime
argument (a MIME type string, such as"image/png"
), which attempt to displayx
using the requested MIME type only, throwing aMethodError
if this type is not supported by either the display(s) or byx
. With these variants, one can also supply the “raw” data in the requested MIME type by passingx::AbstractString
(for MIME types with text-based storage, such as text/html or application/postscript) orx::Vector{UInt8}
(for binary MIME types).
redisplay
(x)¶redisplay
(d::Display, x)redisplay
(mime, x)redisplay
(d::Display, mime, x)By default, the
redisplay
functions simply calldisplay
. However, some display backends may overrideredisplay
to modify an existing display ofx
(if any). Usingredisplay
is also a hint to the backend thatx
may be redisplayed several times, and the backend may choose to defer the display until (for example) the next interactive prompt.
displayable
(mime) → Bool¶displayable
(d::Display, mime) → BoolReturns a boolean value indicating whether the given
mime
type (string) is displayable by any of the displays in the current display stack, or specifically by the displayd
in the second variant.
show
(stream, mime, x)The
display
functions ultimately callshow
in order to write an objectx
as a givenmime
type to a given I/Ostream
(usually a memory buffer), if possible. In order to provide a rich multimedia representation of a user-defined typeT
, it is only necessary to define a newshow
method forT
, via:show(stream,::MIME"mime",x::T)=...
, wheremime
is a MIME-type string and the function body callswrite
(or similar) to write that representation ofx
tostream
. (Note that theMIME""
notation only supports literal strings; to constructMIME
types in a more flexible manner useMIME{Symbol("")}
.)For example, if you define a
MyImage
type and know how to write it to a PNG file, you could define a functionshow(stream,::MIME"image/png",x::MyImage)=...
to allow your images to be displayed on any PNG-capableDisplay
(such as IJulia). As usual, be sure toimportBase.show
in order to add new methods to the built-in Julia functionshow
.The default MIME type is
MIME"text/plain"
. There is a fallback definition fortext/plain
output that callsshow
with 2 arguments. Therefore, this case should be handled by defining a 2-argumentshow(stream::IO,x::MyType)
method.Technically, the
MIME"mime"
macro defines a singleton type for the givenmime
string, which allows us to exploit Julia’s dispatch mechanisms in determining how to display objects of any given type.The first argument to
show
can be anIOContext
specifying output format properties. SeeIOContext
for details.
mimewritable
(mime, x)¶Returns a boolean value indicating whether or not the object
x
can be written as the givenmime
type. (By default, this is determined automatically by the existence of the correspondingshow
function fortypeof(x)
.)
reprmime
(mime, x)¶Returns an
AbstractString
orVector{UInt8}
containing the representation ofx
in the requestedmime
type, as written byshow
(throwing aMethodError
if no appropriateshow
is available). AnAbstractString
is returned for MIME types with textual representations (such as"text/html"
or"application/postscript"
), whereas binary data is returned asVector{UInt8}
. (The functionistextmime(mime)
returns whether or not Julia treats a givenmime
type as text.)As a special case, if
x
is anAbstractString
(for textual MIME types) or aVector{UInt8}
(for binary MIME types), thereprmime
function assumes thatx
is already in the requestedmime
format and simply returnsx
.
stringmime
(mime, x)¶Returns an
AbstractString
containing the representation ofx
in the requestedmime
type. This is similar toreprmime
except that binary data is base64-encoded as an ASCII string.
As mentioned above, one can also define new display backends. For
example, a module that can display PNG images in a window can register
this capability with Julia, so that calling display(x)
on types
with PNG representations will automatically display the image using
the module’s window.
In order to define a new display backend, one should first create a
subtype D
of the abstract class Display
. Then, for each MIME
type (mime
string) that can be displayed on D
, one should
define a function display(d::D,::MIME"mime",x)=...
that
displays x
as that MIME type, usually by calling reprmime(mime,x)
. A MethodError
should be thrown if x
cannot be displayed
as that MIME type; this is automatic if one calls reprmime
.
Finally, one should define a function display(d::D,x)
that
queries mimewritable(mime,x)
for the mime
types supported by
D
and displays the “best” one; a MethodError
should be thrown
if no supported MIME types are found for x
. Similarly, some
subtypes may wish to override redisplay(d::D,...)
. (Again, one
should importBase.display
to add new methods to display
.)
The return values of these functions are up to the implementation
(since in some cases it may be useful to return a display “handle” of
some type). The display functions for D
can then be called
directly, but they can also be invoked automatically from
display(x)
simply by pushing a new display onto the display-backend
stack with:
pushdisplay
(d::Display)¶Pushes a new display
d
on top of the global display-backend stack. Callingdisplay(x)
ordisplay(mime,x)
will displayx
on the topmost compatible backend in the stack (i.e., the topmost backend that does not throw aMethodError
).
popdisplay
()¶popdisplay
(d::Display)Pop the topmost backend off of the display-backend stack, or the topmost copy of
d
in the second variant.
TextDisplay
(stream)¶Returns a
TextDisplay<:Display
, which can display any object as the text/plain MIME type (only), writing the text representation to the given I/O stream. (The text representation is the same as the way an object is printed in the Julia REPL.)
istextmime
(m::MIME)¶Determine whether a MIME type is text data.
Memory-mapped I/O¶
Mmap.
Anonymous
(name, readonly, create)¶Create an
IO
-like object for creating zeroed-out mmapped-memory that is not tied to a file for use inMmap.mmap
. Used bySharedArray
for creating shared memory arrays.
Mmap.
mmap
(io::Union{IOStream,AbstractString,Mmap.AnonymousMmap}[, type::Type{Array{T,N}}, dims, offset]; grow::Bool=true, shared::Bool=true)¶Mmap.
mmap
(type::Type{Array{T, N}}, dims)Create an
Array
whose values are linked to a file, using memory-mapping. This provides a convenient way of working with data too large to fit in the computer’s memory.The type is an
Array{T,N}
with a bits-type element ofT
and dimensionN
that determines how the bytes of the array are interpreted. Note that the file must be stored in binary format, and no format conversions are possible (this is a limitation of operating systems, not Julia).dims
is a tuple or singleInteger
specifying the size or length of the array.The file is passed via the stream argument, either as an open
IOStream
or filename string. When you initialize the stream, use"r"
for a “read-only” array, and"w+"
to create a new array used to write values to disk.If no
type
argument is specified, the default isVector{UInt8}
.Optionally, you can specify an offset (in bytes) if, for example, you want to skip over a header in the file. The default value for the offset is the current stream position for an
IOStream
.The
grow
keyword argument specifies whether the disk file should be grown to accommodate the requested size of array (if the total file size is < requested array size). Write privileges are required to grow the file.The
shared
keyword argument specifies whether the resultingArray
and changes made to it will be visible to other processes mapping the same file.For example, the following code
# Create a file for mmapping# (you could alternatively use mmap to do this step, too)A=rand(1:20,5,30)s=open("/tmp/mmap.bin","w+")# We'll write the dimensions of the array as the first two Ints in the filewrite(s,size(A,1))write(s,size(A,2))# Now write the datawrite(s,A)close(s)# Test by reading it back ins=open("/tmp/mmap.bin")# default is read-onlym=read(s,Int)n=read(s,Int)A2=Mmap.mmap(s,Matrix{Int},(m,n))
creates a
m
-by-n
Matrix{Int}
, linked to the file associated with streams
.A more portable file would need to encode the word size – 32 bit or 64 bit – and endianness information in the header. In practice, consider encoding binary data using standard formats like HDF5 (which can be used with memory-mapping).
Mmap.
mmap
(io, BitArray[, dims, offset])Create a
BitArray
whose values are linked to a file, using memory-mapping; it has the same purpose, works in the same way, and has the same arguments, asmmap()
, but the byte representation is different.Example:
B=Mmap.mmap(s,BitArray,(25,30000))
This would create a 25-by-30000
BitArray
, linked to the file associated with streams
.
Mmap.
sync!
(array)¶Forces synchronization between the in-memory version of a memory-mapped
Array
orBitArray
and the on-disk version.
Network I/O¶
connect
([host, ]port) → TCPSocket¶Connect to the host
host
on portport
.
connect
(path) → PipeEndpointConnect to the named pipe / UNIX domain socket at
path
.
listen
([addr, ]port) → TCPServer¶Listen on port on the address specified by
addr
. By default this listens on localhost only. To listen on all interfaces passIPv4(0)
orIPv6(0)
as appropriate.
listen
(path) → PipeServerCreate and listen on a named pipe / UNIX domain socket.
getaddrinfo
(host)¶Gets the IP address of the
host
(may have to do a DNS lookup)
getsockname
(sock::Union{TCPServer, TCPSocket}) → (IPAddr,UInt16)¶Get the IP address and the port that the given TCP socket is connected to (or bound to, in the case of TCPServer).
IPv4
(host::Integer) → IPv4¶Returns IPv4 object from ip address formatted as Integer.
IPv6
(host::Integer) → IPv6¶Returns IPv6 object from ip address formatted as Integer
nb_available
(stream)¶Returns the number of bytes available for reading before a read from this stream or buffer will block.
accept
(server[, client])¶Accepts a connection on the given server and returns a connection to the client. An uninitialized client stream may be provided, in which case it will be used instead of creating a new stream.
listenany
(port_hint) → (UInt16,TCPServer)¶Create a
TCPServer
on any port, using hint as a starting point. Returns a tuple of the actual port that the server was created on and the server itself.
poll_fd
(fd, timeout_s::Real; readable=false, writable=false)¶Monitor a file descriptor
fd
for changes in the read or write availability, and with a timeout given bytimeout_s
seconds.The keyword arguments determine which of read and/or write status should be monitored; at least one of them must be set to
true
.The returned value is an object with boolean fields
readable
,writable
, andtimedout
, giving the result of the polling.
poll_file
(path, interval_s::Real, timeout_s::Real) → (previous::StatStruct, current::StatStruct)¶Monitor a file for changes by polling every
interval_s
seconds until a change occurs ortimeout_s
seconds have elapsed. Theinterval_s
should be a long period; the default is 5.007 seconds.Returns a pair of
StatStruct
objects(previous,current)
when a change is detected.To determine when a file was modified, compare
mtime(prev)!=mtime(current)
to detect notification of changes. However, usingwatch_file
for this operation is preferred, since it is more reliable and efficient, although in some situations it may not be available.
watch_file
(path, timeout_s::Real)¶Watch file or directory
path
for changes until a change occurs ortimeout_s
seconds have elapsed.The returned value is an object with boolean fields
changed
,renamed
, andtimedout
, giving the result of watching the file.This behavior of this function varies slightly across platforms. See <https://nodejs.org/api/fs.html#fs_caveats> for more detailed information.
bind
(socket::Union{UDPSocket, TCPSocket}, host::IPAddr, port::Integer; ipv6only=false)¶Bind
socket
to the givenhost:port
. Note that0.0.0.0
will listen on all devices.ipv6only
parameter disables dual stack mode. If it’strue
, only IPv6 stack is created.
send
(socket::UDPSocket, host::IPv4, port::Integer, msg)¶Send
msg
oversocket
tohost:port
.
recv
(socket::UDPSocket)¶Read a UDP packet from the specified socket, and return the bytes received. This call blocks.
recvfrom
(socket::UDPSocket) → (address, data)¶Read a UDP packet from the specified socket, returning a tuple of (address, data), where address will be either IPv4 or IPv6 as appropriate.
setopt
(sock::UDPSocket; multicast_loop = nothing, multicast_ttl=nothing, enable_broadcast=nothing, ttl=nothing)¶Set UDP socket options.
multicast_loop
: loopback for multicast packets (default:true
).multicast_ttl
: TTL for multicast packets.enable_broadcast
: flag must be set totrue
if socket will be used for broadcast messages, or else the UDP system will return an access error (default:false
).ttl
: Time-to-live of packets sent on the socket.
ntoh
(x)¶Converts the endianness of a value from Network byte order (big-endian) to that used by the Host.
hton
(x)¶Converts the endianness of a value from that used by the Host to Network byte order (big-endian).
ltoh
(x)¶Converts the endianness of a value from Little-endian to that used by the Host.
htol
(x)¶Converts the endianness of a value from that used by the Host to Little-endian.
ENDIAN_BOM
¶The 32-bit byte-order-mark indicates the native byte order of the host machine. Little-endian machines will contain the value
0x04030201
. Big-endian machines will contain the value0x01020304
.