# class CSV

This class provides a complete interface to `CSV` files and data. It offers tools to enable you to read and write to and from Strings or `IO` objects, as needed.

## Reading¶ ↑

### From a `File`¶ ↑

#### A Line at a `Time`¶ ↑

    CSV.foreach("path/to/file.csv") do |row|
      # use row here...
    end

#### All at Once¶ ↑

    arr_of_arrs = CSV.read("path/to/file.csv")

### From a String¶ ↑

#### A Line at a `Time`¶ ↑

    CSV.parse("CSV,data,String") do |row|
      # use row here...
    end

#### All at Once¶ ↑

    arr_of_arrs = CSV.parse("CSV,data,String")

## Writing¶ ↑

### To a `File`¶ ↑

    CSV.open("path/to/file.csv", "wb") do |csv|
      csv << ["row", "of", "CSV", "data"]
      csv << ["another", "row"]
      # ...
    end

### To a String¶ ↑

    csv_string = CSV.generate do |csv|
      csv << ["row", "of", "CSV", "data"]
      csv << ["another", "row"]
      # ...
    end

## Convert a Single Line¶ ↑

    csv_string = ["CSV", "data"].to_csv   # to CSV
    csv_array  = "CSV,String".parse_csv   # from CSV

## Shortcut Interface¶ ↑

    CSV             { |csv_out| csv_out << %w{my data here} }  # to $stdout
    CSV(csv = "")   { |csv_str| csv_str << %w{my data here} }  # to a String
    CSV($stderr)    { |csv_err| csv_err << %w{my data here} }  # to $stderr
    CSV($stdin)     { |csv_in|  csv_in.each { |row| p row } }  # from $stdin

## Advanced Usage¶ ↑

### Wrap an `IO` `Object`¶ ↑

    csv = CSV.new(io, options)
    # ... read (with gets() or each()) from and write (with <<) to csv here ...

## `CSV` and Character Encodings (M17n or Multilingualization)¶ ↑

This new `CSV` parser is m17n savvy. The parser works in the `Encoding` of the `IO` or String object being read from or written to. Your data is never transcoded (unless you ask Ruby to transcode it for you) and will literally be parsed in the `Encoding` it is in. Thus `CSV` will return Arrays or Rows of Strings in the `Encoding` of your data. This is accomplished by transcoding the parser itself into your `Encoding`.

Some transcoding must take place, of course, to accomplish this multiencoding support. For example, `:col_sep`, `:row_sep`, and `:quote_char` must be transcoded to match your data. Hopefully this makes the entire process feel transparent, since CSV's defaults should just magically work for your data. However, you can set these values manually in the target `Encoding` to avoid the translation.

It's also important to note that while all of CSV's core parser is now `Encoding` agnostic, some features are not. For example, the built-in converters will try to transcode data to UTF-8 before making conversions. Again, you can provide custom converters that are aware of your Encodings to avoid this translation. It's just too hard for me to support native conversions in all of Ruby's Encodings.

Anyway, the practical side of this is simple: make sure `IO` and String objects passed into `CSV` have the proper `Encoding` set and everything should just work. `CSV` methods that allow you to open `IO` objects (`CSV::foreach()`, `CSV::open()`, `CSV::read()`, and `CSV::readlines()`) do allow you to specify the `Encoding`.

One minor exception comes when generating `CSV` into a String with an `Encoding` that is not ASCII compatible. There's no existing data for `CSV` to use to prepare itself and thus you will probably need to manually specify the desired `Encoding` for most of those cases. It will try to guess using the fields in a row of output though, when using `CSV::generate_line()` or Array#to_csv().

I try to point out any other `Encoding` issues in the documentation of methods as they come up.

This has been tested to the best of my ability with all non-“dummy” Encodings Ruby ships with. However, it is brave new code and may have some bugs. Please feel free to report any issues you find with it.

### Constants

ConverterEncoding

  The encoding used by all converters.

Converters

  This `Hash` holds the built-in converters of `CSV` that can be accessed by name. You can select `Converters` with `CSV.convert()` or through the `options` `Hash` passed to `CSV::new()`.

  `:integer`

    Converts any field Integer() accepts.

  `:float`

    Converts any field Float() accepts.

  `:numeric`

    A combination of `:integer` and `:float`.

  `:date`

    Converts any field `Date::parse()` accepts.

  `:date_time`

    Converts any field `DateTime::parse()` accepts.

  `:all`

    All built-in converters. A combination of `:date_time` and `:numeric`.

  All built-in converters transcode field data to UTF-8 before attempting a conversion. If your data cannot be transcoded to UTF-8 the conversion will fail and the field will remain unchanged.

  This `Hash` is intentionally left unfrozen and users should feel free to add values to it that can be accessed by all `CSV` objects.

  To add a combo field, the value should be an Array of names. Combo fields can be nested with other combo fields.

DEFAULT_OPTIONS

  The options used when no overrides are given by calling code. They are:

  `:col_sep`

    `","`

  `:row_sep`

    `:auto`

  `:quote_char`

    `'"'`

  `:field_size_limit`

    `nil`

  `:converters`

    `nil`

  `:unconverted_fields`

    `nil`

  `:headers`

    `false`

  `:return_headers`

    `false`

  `:header_converters`

    `nil`

  `:skip_blanks`

    `false`

  `:force_quotes`

    `false`

  `:skip_lines`

    `nil`

  `:liberal_parsing`

    `false`

DateMatcher

  A `Regexp` used to find and convert some common `Date` formats.

DateTimeMatcher

  A `Regexp` used to find and convert some common `DateTime` formats.

FieldInfo

  A `FieldInfo` `Struct` contains details about a field's position in the data source it was read from. `CSV` will pass this `Struct` to some blocks that make decisions based on field structure. See `CSV.convert_fields()` for an example.

  `index`

    The zero-based index of the field in its row.

  `line`

    The line of the data source this row is from.

  `header`

    The header for the column, when available.

HeaderConverters

  This `Hash` holds the built-in header converters of `CSV` that can be accessed by name. You can select `HeaderConverters` with `CSV.header_convert()` or through the `options` `Hash` passed to `CSV::new()`.

  `:downcase`

    Calls downcase() on the header String.

  `:symbol`

    Leading/trailing spaces are dropped, string is downcased, remaining spaces are replaced with underscores, non-word characters are dropped, and finally to_sym() is called.

  All built-in header converters transcode header data to UTF-8 before attempting a conversion. If your data cannot be transcoded to UTF-8 the conversion will fail and the header will remain unchanged.

  This `Hash` is intentionally left unfrozen and users should feel free to add values to it that can be accessed by all `CSV` objects.

  To add a combo field, the value should be an Array of names. Combo fields can be nested with other combo fields.

VERSION

  The version of the installed library.

### Attributes

col_sep[R]

The encoded `:col_sep` used in parsing and writing. See `CSV::new` for details.

encoding[R]

The `Encoding` `CSV` is parsing or writing in. This will be the `Encoding` you receive parsed data in and/or the `Encoding` data will be written in.

field_size_limit[R]

The limit for field size, if any. See `CSV::new` for details.

lineno[R]

The line number of the last row read from this file. Fields with nested line-end characters will not affect this count.

quote_char[R]

The encoded `:quote_char` used in parsing and writing. See `CSV::new` for details.

row_sep[R]

The encoded `:row_sep` used in parsing and writing. See `CSV::new` for details.

skip_lines[R]

The regex marking a line as a comment. See `CSV::new` for details

### Public Class Methods

filter( options = Hash.new ) { |row| ... } click to toggle source
filter( input, options = Hash.new ) { |row| ... }
filter( input, output, options = Hash.new ) { |row| ... }

This method is a convenience for building Unix-like filters for `CSV` data. Each row is yielded to the provided block which can alter it as needed. After the block returns, the row is appended to `output` altered or not.

The `input` and `output` arguments can be anything `CSV::new()` accepts (generally String or `IO` objects). If not given, they default to `ARGF` and `$stdout`.

The `options` parameter is also filtered down to `CSV::new()` after some clever key parsing. Any key beginning with `:in_` or `:input_` will have that leading identifier stripped and will only be used in the `options` `Hash` for the `input` object. Keys starting with `:out_` or `:output_` affect only `output`. All other keys are assigned to both objects.

The `:output_row_sep` `option` defaults to `$INPUT_RECORD_SEPARATOR` (`$/`).

    # File lib/csv.rb, line 1104
    def self.filter(*args)
      # parse options for input, output, or both
      in_options, out_options = Hash.new, {row_sep: $INPUT_RECORD_SEPARATOR}
      if args.last.is_a? Hash
        args.pop.each do |key, value|
          case key.to_s
          when /\Ain(?:put)?_(.+)\Z/
            in_options[$1.to_sym] = value
          when /\Aout(?:put)?_(.+)\Z/
            out_options[$1.to_sym] = value
          else
            in_options[key]  = value
            out_options[key] = value
          end
        end
      end
      # build input and output wrappers
      input  = new(args.shift || ARGF,    in_options)
      output = new(args.shift || $stdout, out_options)

      # read, yield, write
      input.each do |row|
        yield row
        output << row
      end
    end

foreach(path, options = Hash.new, &block) click to toggle source

This method is intended as the primary interface for reading `CSV` files. You pass a `path` and any `options` you wish to set for the read. Each row of file will be passed to the provided `block` in turn.

The `options` parameter can be anything `CSV::new()` understands. This method also understands an additional `:encoding` parameter that you can use to specify the `Encoding` of the data in the file to be read. You must provide this unless your data is in `Encoding::default_external()`. `CSV` will use this to determine how to parse the data. You may provide a second `Encoding` to have the data transcoded as it is read. For example, `encoding: "UTF-32BE:UTF-8"` would read UTF-32BE data from the file but transcode it to UTF-8 before `CSV` parses it.

    # File lib/csv.rb, line 1145
    def self.foreach(path, options = Hash.new, &block)
      return to_enum(__method__, path, options) unless block
      open(path, options) do |csv|
        csv.each(&block)
      end
    end

generate( str, options = Hash.new ) { |csv| ... } click to toggle source
generate( options = Hash.new ) { |csv| ... }

This method wraps a String you provide, or an empty default String, in a `CSV` object which is passed to the provided block. You can use the block to append `CSV` rows to the String and when the block exits, the final String will be returned.

Note that a passed String is modified by this method. Call dup() before passing if you need a new String.

The `options` parameter can be anything `CSV::new()` understands. This method understands an additional `:encoding` parameter when not passed a String to set the base `Encoding` for the output. `CSV` needs this hint if you plan to output non-ASCII compatible data.

    # File lib/csv.rb, line 1170
    def self.generate(*args)
      # add a default empty String, if none was given
      if args.first.is_a? String
        io = StringIO.new(args.shift)
        io.seek(0, IO::SEEK_END)
        args.unshift(io)
      else
        encoding = args[-1][:encoding] if args.last.is_a?(Hash)
        str      = String.new
        str.force_encoding(encoding) if encoding
        args.unshift(str)
      end
      csv = new(*args)  # wrap
      yield csv         # yield for appending
      csv.string        # return final String
    end

generate_line(row, options = Hash.new) click to toggle source

This method is a shortcut for converting a single row (Array) into a `CSV` String.

The `options` parameter can be anything `CSV::new()` understands. This method understands an additional `:encoding` parameter to set the base `Encoding` for the output. This method will try to guess your `Encoding` from the first non-`nil` field in `row`, if possible, but you may need to use this parameter as a backup plan.

The `:row_sep` `option` defaults to `$INPUT_RECORD_SEPARATOR` (`$/`) when calling this method.

    # File lib/csv.rb, line 1200
    def self.generate_line(row, options = Hash.new)
      options  = {row_sep: $INPUT_RECORD_SEPARATOR}.merge(options)
      encoding = options.delete(:encoding)
      str      = String.new
      if encoding
        str.force_encoding(encoding)
      elsif field = row.find { |f| not f.nil? }
        str.force_encoding(String(field).encoding)
      end
      (new(str, options) << row).string
    end

instance(data = $stdout, options = Hash.new) { |instance| ... } click to toggle source

This method will return a `CSV` instance, just like `CSV::new()`, but the instance will be cached and returned for all future calls to this method for the same `data` object (tested by `Object#object_id()`) with the same `options`.

If a block is given, the instance is passed to the block and the return value becomes the return value of the block.

    # File lib/csv.rb, line 1064
    def self.instance(data = $stdout, options = Hash.new)
      # create a _signature_ for this method call, data object and options
      sig = [data.object_id] +
            options.values_at(*DEFAULT_OPTIONS.keys.sort_by { |sym| sym.to_s })

      # fetch or create the instance for this signature
      @@instances ||= Hash.new
      instance    =   (@@instances[sig] ||= new(data, options))

      if block_given?
        yield instance  # run block, if given, returning result
      else
        instance        # or return the instance
      end
    end

new(data, options = Hash.new) click to toggle source

This constructor will wrap either a String or `IO` object passed in `data` for reading and/or writing. In addition to the `CSV` instance methods, several `IO` methods are delegated. (See `CSV::open()` for a complete list.) If you pass a String for `data`, you can later retrieve it (after writing to it, for example) with CSV.string().

Note that a wrapped String will be positioned at the beginning (for reading). If you want it at the end (for writing), use `CSV::generate()`. If you want any other positioning, pass a preset `StringIO` object instead.

You may set any reading and/or writing preferences in the `options` `Hash`. Available options are:

`:col_sep`

  The String placed between each field. This String will be transcoded into the data's `Encoding` before parsing.

`:row_sep`

  The String appended to the end of each row. This can be set to the special `:auto` setting, which requests that `CSV` automatically discover this from the data. Auto-discovery reads ahead in the data looking for the next `"\r\n"`, `"\n"`, or `"\r"` sequence. A sequence will be selected even if it occurs in a quoted field, assuming that you would have the same line endings there. If none of those sequences is found, `data` is `ARGF`, `STDIN`, `STDOUT`, or `STDERR`, or the stream is only available for output, the default `$INPUT_RECORD_SEPARATOR` (`$/`) is used. Obviously, discovery takes a little time. `Set` manually if speed is important. Also note that `IO` objects should be opened in binary mode on Windows if this feature will be used as the line-ending translation can cause problems with resetting the document position to where it was before the read ahead. This String will be transcoded into the data's `Encoding` before parsing.

`:quote_char`

  The character used to quote fields. This has to be a single character String. This is useful for application that incorrectly use `'` as the quote character instead of the correct `"`. `CSV` will always consider a double sequence of this character to be an escaped quote. This String will be transcoded into the data's `Encoding` before parsing.

`:field_size_limit`

  This is a maximum size `CSV` will read ahead looking for the closing quote for a field. (In truth, it reads to the first line ending beyond this size.) If a quote cannot be found within the limit `CSV` will raise a `MalformedCSVError`, assuming the data is faulty. You can use this limit to prevent what are effectively DoS attacks on the parser. However, this limit can cause a legitimate parse to fail and thus is set to `nil`, or off, by default.

`:converters`

  An Array of names from the `Converters` `Hash` and/or lambdas that handle custom conversion. A single converter doesn't have to be in an Array. All built-in converters try to transcode fields to UTF-8 before converting. The conversion will fail if the data cannot be transcoded, leaving the field unchanged.

`:unconverted_fields`

  If set to `true`, an unconverted_fields() method will be added to all returned rows (Array or `CSV::Row`) that will return the fields as they were before conversion. Note that `:headers` supplied by Array or String were not fields of the document and thus will have an empty Array attached.

`:headers`

  If set to `:first_row` or `true`, the initial row of the `CSV` file will be treated as a row of headers. If set to an Array, the contents will be used as the headers. If set to a String, the String is run through a call of `CSV::parse_line()` with the same `:col_sep`, `:row_sep`, and `:quote_char` as this instance to produce an Array of headers. This setting causes `CSV#shift()` to return rows as `CSV::Row` objects instead of Arrays and `CSV#read()` to return `CSV::Table` objects instead of an Array of Arrays.

`:return_headers`

  When `false`, header rows are silently swallowed. If set to `true`, header rows are returned in a `CSV::Row` object with identical headers and fields (save that the fields do not go through the converters).

`:write_headers`

  When `true` and `:headers` is set, a header row will be added to the output.

`:header_converters`

  Identical in functionality to `:converters` save that the conversions are only made to header rows. All built-in converters try to transcode headers to UTF-8 before converting. The conversion will fail if the data cannot be transcoded, leaving the header unchanged.

`:skip_blanks`

  When set to a `true` value, `CSV` will skip over any empty rows. Note that this setting will not skip rows that contain column separators, even if the rows contain no actual data. If you want to skip rows that contain separators but no content, consider using `:skip_lines`, or inspecting fields.compact.empty? on each row.

`:force_quotes`

  When set to a `true` value, `CSV` will quote all `CSV` fields it creates.

`:skip_lines`

  When set to an object responding to `match`, every line matching it is considered a comment and ignored during parsing. When set to a String, it is first converted to a `Regexp`. When set to `nil` no line is considered a comment. If the passed object does not respond to `match`, `ArgumentError` is thrown.

`:liberal_parsing`

  When set to a `true` value, `CSV` will attempt to parse input not conformant with RFC 4180, such as double quotes in unquoted fields.

See `CSV::DEFAULT_OPTIONS` for the default settings.

Options cannot be overridden in the instance methods for performance reasons, so be sure to set what you want here.

    # File lib/csv.rb, line 1529
    def initialize(data, options = Hash.new)
      if data.nil?
        raise ArgumentError.new("Cannot parse nil as CSV")
      end

      # build the options for this read/write
      options = DEFAULT_OPTIONS.merge(options)

      # create the IO object we will read from
      @io       = data.is_a?(String) ? StringIO.new(data) : data
      # honor the IO encoding if we can, otherwise default to ASCII-8BIT
      @encoding = raw_encoding(nil) ||
                  ( if encoding = options.delete(:internal_encoding)
                      case encoding
                      when Encoding; encoding
                      else Encoding.find(encoding)
                      end
                    end ) ||
                  ( case encoding = options.delete(:encoding)
                    when Encoding; encoding
                    when /\A[^:]+/; Encoding.find($&)
                    end ) ||
                  Encoding.default_internal || Encoding.default_external
      #
      # prepare for building safe regular expressions in the target encoding,
      # if we can transcode the needed characters
      #
      @re_esc   =   "\\".encode(@encoding).freeze rescue ""
      @re_chars =   /#{%"[-\\]\\[\\.^$?*+{}()|# \r\n\t\f\v]".encode(@encoding)}/

      init_separators(options)
      init_parsers(options)
      init_converters(options)
      init_headers(options)
      init_comments(options)

      @force_encoding = !!(encoding || options.delete(:encoding))
      options.delete(:internal_encoding)
      options.delete(:external_encoding)
      unless options.empty?
        raise ArgumentError, "Unknown options:  #{options.keys.join(', ')}."
      end

      # track our own lineno since IO gets confused about line-ends is CSV fields
      @lineno = 0
    end

open( filename, mode = "rb", options = Hash.new ) { |faster_csv| ... } click to toggle source
open( filename, options = Hash.new ) { |faster_csv| ... }
open( filename, mode = "rb", options = Hash.new )
open( filename, options = Hash.new )

This method opens an `IO` object, and wraps that with `CSV`. This is intended as the primary interface for writing a `CSV` file.

You must pass a `filename` and may optionally add a `mode` for Ruby's open(). You may also pass an optional `Hash` containing any `options` `CSV::new()` understands as the final argument.

This method works like Ruby's open() call, in that it will pass a `CSV` object to a provided block and close it when the block terminates, or it will return the `CSV` object when no block is provided. (Note: This is different from the Ruby 1.8 `CSV` library which passed rows to the block. Use `CSV::foreach()` for that behavior.)

You must provide a `mode` with an embedded `Encoding` designator unless your data is in `Encoding::default_external()`. `CSV` will check the `Encoding` of the underlying `IO` object (set by the `mode` you pass) to determine how to parse the data. You may provide a second `Encoding` to have the data transcoded as it is read just as you can with a normal call to `IO::open()`. For example, `"rb:UTF-32BE:UTF-8"` would read UTF-32BE data from the file but transcode it to UTF-8 before `CSV` parses it.

An opened `CSV` object will delegate to many `IO` methods for convenience. You may call:

- binmode()

- binmode?()

- close()

- close_read()

- close_write()

- closed?()

- eof()

- eof?()

- external_encoding()

- fcntl()

- fileno()

- flock()

- flush()

- fsync()

- internal_encoding()

- ioctl()

- isatty()

- path()

- pid()

- pos()

- pos=()

- reopen()

- seek()

- stat()

- sync()

- sync=()

- tell()

- to_i()

- to_io()

- truncate()

- tty?()

    # File lib/csv.rb, line 1275
    def self.open(*args)
      # find the +options+ Hash
      options = if args.last.is_a? Hash then args.pop else Hash.new end
      # wrap a File opened with the remaining +args+ with no newline
      # decorator
      file_opts = {universal_newline: false}.merge(options)
      begin
        f = File.open(*args, file_opts)
      rescue ArgumentError => e
        raise unless /needs binmode/ =~ e.message and args.size == 1
        args << "rb"
        file_opts = {encoding: Encoding.default_external}.merge(file_opts)
        retry
      end
      begin
        csv = new(f, options)
      rescue Exception
        f.close
        raise
      end

      # handle blocks like Ruby's open(), not like the CSV library
      if block_given?
        begin
          yield csv
        ensure
          csv.close
        end
      else
        csv
      end
    end

parse( str, options = Hash.new ) { |row| ... } click to toggle source
parse( str, options = Hash.new )

This method can be used to easily parse `CSV` out of a String. You may either provide a `block` which will be called with each row of the String in turn, or just use the returned Array of Arrays (when no `block` is given).

You pass your `str` to read from, and an optional `options` `Hash` containing anything `CSV::new()` understands.

    # File lib/csv.rb, line 1320
    def self.parse(*args, &block)
      csv = new(*args)
      if block.nil?  # slurp contents, if no block is given
        begin
          csv.read
        ensure
          csv.close
        end
      else           # or pass each row to a provided block
        csv.each(&block)
      end
    end

parse_line(line, options = Hash.new) click to toggle source

This method is a shortcut for converting a single line of a `CSV` String into an Array. Note that if `line` contains multiple rows, anything beyond the first row is ignored.

The `options` parameter can be anything `CSV::new()` understands.

    # File lib/csv.rb, line 1340
    def self.parse_line(line, options = Hash.new)
      new(line, options).shift
    end

read(path, *options) click to toggle source

Use to slurp a `CSV` file into an Array of Arrays. Pass the `path` to the file and any `options` `CSV::new()` understands. This method also understands an additional `:encoding` parameter that you can use to specify the `Encoding` of the data in the file to be read. You must provide this unless your data is in `Encoding::default_external()`. `CSV` will use this to determine how to parse the data. You may provide a second `Encoding` to have the data transcoded as it is read. For example, `encoding: "UTF-32BE:UTF-8"` would read UTF-32BE data from the file but transcode it to UTF-8 before `CSV` parses it.

    # File lib/csv.rb, line 1355
    def self.read(path, *options)
      open(path, *options) { |csv| csv.read }
    end

readlines(*args) click to toggle source

Alias for `CSV::read()`.

    # File lib/csv.rb, line 1360
    def self.readlines(*args)
      read(*args)
    end

table(path, options = Hash.new) click to toggle source

A shortcut for:

    CSV.read( path, { headers:           true,
                      converters:        :numeric,
                      header_converters: :symbol }.merge(options) )

    # File lib/csv.rb, line 1371
    def self.table(path, options = Hash.new)
      read( path, { headers:           true,
                    converters:        :numeric,
                    header_converters: :symbol }.merge(options) )
    end

### Public Instance Methods

<<(row) click to toggle source

The primary write method for wrapped Strings and IOs, `row` (an Array or `CSV::Row`) is converted to `CSV` and appended to the data source. When a `CSV::Row` is passed, only the row's fields() are appended to the output.

The data source must be open for writing.

    # File lib/csv.rb, line 1688
    def <<(row)
      # make sure headers have been assigned
      if header_row? and [Array, String].include? @use_headers.class
        parse_headers  # won't read data for Array or String
        self << @headers if @write_headers
      end

      # handle CSV::Row objects and Hashes
      row = case row
            when self.class::Row then row.fields
            when Hash            then @headers.map { |header| row[header] }
            else                      row
            end

      @headers =  row if header_row?
      @lineno  += 1

      output = row.map(&@quote).join(@col_sep) + @row_sep  # quote and separate
      if @io.is_a?(StringIO)             and
         output.encoding != (encoding = raw_encoding)
        if @force_encoding
          output = output.encode(encoding)
        elsif (compatible_encoding = Encoding.compatible?(@io.string, output))
          @io.set_encoding(compatible_encoding)
          @io.seek(0, IO::SEEK_END)
        end
      end
      @io << output

      self  # for chaining
    end

Also aliased as: add_row, puts
add_row(row)
Alias for: <<
convert( name ) click to toggle source
convert { |field| ... }
convert { |field, field_info| ... }

You can use this method to install a `CSV::Converters` built-in, or provide a block that handles a custom conversion.

If you provide a block that takes one argument, it will be passed the field and is expected to return the converted value or the field itself. If your block takes two arguments, it will also be passed a `CSV::FieldInfo` `Struct`, containing details about the field. Again, the block should return a converted field or the field itself.

    # File lib/csv.rb, line 1737
    def convert(name = nil, &converter)
      add_converter(:converters, self.class::Converters, name, &converter)
    end

converters() click to toggle source

Returns the current list of converters in effect. See `CSV::new` for details. Built-in converters will be returned by name, while others will be returned as is.

    # File lib/csv.rb, line 1602
    def converters
      @converters.map do |converter|
        name = Converters.rassoc(converter)
        name ? name.first : converter
      end
    end

each() { |row| ... } click to toggle source

Yields each row of the data source in turn.

Support for `Enumerable`.

The data source must be open for reading.

    # File lib/csv.rb, line 1768
    def each
      if block_given?
        while row = shift
          yield row
        end
      else
        to_enum
      end
    end

force_quotes?() click to toggle source

Returns `true` if all output fields are quoted. See `CSV::new` for details.

    # File lib/csv.rb, line 1645
    def force_quotes?()       @force_quotes       end

gets()
Alias for: shift
header_convert( name ) click to toggle source
header_convert { |field| ... }
header_convert { |field, field_info| ... }

Identical to `CSV#convert()`, but for header rows.

Note that this method must be called before header rows are read to have any effect.

    # File lib/csv.rb, line 1752
    def header_convert(name = nil, &converter)
      add_converter( :header_converters,
                     self.class::HeaderConverters,
                     name,
                     &converter )
    end

header_converters() click to toggle source

Returns the current list of converters in effect for headers. See `CSV::new` for details. Built-in converters will be returned by name, while others will be returned as is.

    # File lib/csv.rb, line 1633
    def header_converters
      @header_converters.map do |converter|
        name = HeaderConverters.rassoc(converter)
        name ? name.first : converter
      end
    end

header_row?() click to toggle source

Returns `true` if the next row read will be a header row.

    # File lib/csv.rb, line 1794
    def header_row?
      @use_headers and @headers.nil?
    end

headers() click to toggle source

Returns `nil` if headers will not be used, `true` if they will but have not yet been read, or the actual headers after they have been read. See `CSV::new` for details.

    # File lib/csv.rb, line 1618
    def headers
      @headers || true if @use_headers
    end

inspect() click to toggle source

Returns a simplified description of the key `CSV` attributes in an ASCII compatible String.

    # File lib/csv.rb, line 1962
    def inspect
      str = ["<#", self.class.to_s, " io_type:"]
      # show type of wrapped IO
      if    @io == $stdout then str << "$stdout"
      elsif @io == $stdin  then str << "$stdin"
      elsif @io == $stderr then str << "$stderr"
      else                      str << @io.class.to_s
      end
      # show IO.path(), if available
      if @io.respond_to?(:path) and (p = @io.path)
        str << " io_path:" << p.inspect
      end
      # show encoding
      str << " encoding:" << @encoding.name
      # show other attributes
      %w[ lineno     col_sep     row_sep
          quote_char skip_blanks liberal_parsing ].each do |attr_name|
        if a = instance_variable_get("@#{attr_name}")
          str << " " << attr_name << ":" << a.inspect
        end
      end
      if @use_headers
        str << " headers:" << headers.inspect
      end
      str << ">"
      begin
        str.join('')
      rescue  # any encoding error
        str.map do |s|
          e = Encoding::Converter.asciicompat_encoding(s.encoding)
          e ? s.encode(e) : s.force_encoding("ASCII-8BIT")
        end.join('')
      end
    end

liberal_parsing?() click to toggle source

Returns `true` if illegal input is handled. See `CSV::new` for details.

    # File lib/csv.rb, line 1647
    def liberal_parsing?()    @liberal_parsing    end

puts(row)
Alias for: <<
read() click to toggle source

Slurps the remaining rows and returns an Array of Arrays.

The data source must be open for reading.

    # File lib/csv.rb, line 1783
    def read
      rows = to_a
      if @use_headers
        Table.new(rows)
      else
        rows
      end
    end

Also aliased as: readlines
readline()
Alias for: shift
readlines()
Alias for: read
return_headers?() click to toggle source

Returns `true` if headers will be returned as a row of results. See `CSV::new` for details.

    # File lib/csv.rb, line 1625
    def return_headers?()     @return_headers     end

rewind() click to toggle source

Rewinds the underlying `IO` object and resets CSV's lineno() counter.

    # File lib/csv.rb, line 1672
    def rewind
      @headers = nil
      @lineno  = 0

      @io.rewind
    end

shift() click to toggle source

The primary read method for wrapped Strings and IOs, a single row is pulled from the data source, parsed and returned as an Array of fields (if header rows are not used) or a `CSV::Row` (when header rows are used).

The data source must be open for reading.

    # File lib/csv.rb, line 1805
    def shift
      #########################################################################
      ### This method is purposefully kept a bit long as simple conditional ###
      ### checks are faster than numerous (expensive) method calls.         ###
      #########################################################################

      # handle headers not based on document content
      if header_row? and @return_headers and
         [Array, String].include? @use_headers.class
        if @unconverted_fields
          return add_unconverted_fields(parse_headers, Array.new)
        else
          return parse_headers
        end
      end

      #
      # it can take multiple calls to <tt>@io.gets()</tt> to get a full line,
      # because of \r and/or \n characters embedded in quoted fields
      #
      in_extended_col = false
      csv             = Array.new

      loop do
        # add another read to the line
        unless parse = @io.gets(@row_sep)
          return nil
        end

        parse.sub!(@parsers[:line_end], "")

        if csv.empty?
          #
          # I believe a blank line should be an <tt>Array.new</tt>, not Ruby 1.8
          # CSV's <tt>[nil]</tt>
          #
          if parse.empty?
            @lineno += 1
            if @skip_blanks
              next
            elsif @unconverted_fields
              return add_unconverted_fields(Array.new, Array.new)
            elsif @use_headers
              return self.class::Row.new(Array.new, Array.new)
            else
              return Array.new
            end
          end
        end

        next if @skip_lines and @skip_lines.match parse

        parts =  parse.split(@col_sep, -1)
        if parts.empty?
          if in_extended_col
            csv[-1] << @col_sep   # will be replaced with a @row_sep after the parts.each loop
          else
            csv << nil
          end
        end

        # This loop is the hot path of csv parsing. Some things may be non-dry
        # for a reason. Make sure to benchmark when refactoring.
        parts.each do |part|
          if in_extended_col
            # If we are continuing a previous column
            if part[-1] == @quote_char && part.count(@quote_char) % 2 != 0
              # extended column ends
              csv[-1] = csv[-1].push(part[0..-2]).join("")
              if csv.last =~ @parsers[:stray_quote]
                raise MalformedCSVError,
                      "Missing or stray quote in line #{lineno + 1}"
              end
              csv.last.gsub!(@quote_char * 2, @quote_char)
              in_extended_col = false
            else
              csv.last.push(part, @col_sep)
            end
          elsif part[0] == @quote_char
            # If we are starting a new quoted column
            if part.count(@quote_char) % 2 != 0
              # start an extended column
              csv << [part[1..-1], @col_sep]
              in_extended_col =  true
            elsif part[-1] == @quote_char
              # regular quoted column
              csv << part[1..-2]
              if csv.last =~ @parsers[:stray_quote]
                raise MalformedCSVError,
                      "Missing or stray quote in line #{lineno + 1}"
              end
              csv.last.gsub!(@quote_char * 2, @quote_char)
            elsif @liberal_parsing
              csv << part
            else
              raise MalformedCSVError,
                    "Missing or stray quote in line #{lineno + 1}"
            end
          elsif part =~ @parsers[:quote_or_nl]
            # Unquoted field with bad characters.
            if part =~ @parsers[:nl_or_lf]
              raise MalformedCSVError, "Unquoted fields do not allow " +
                                       "\\r or \\n (line #{lineno + 1})."
            else
              if @liberal_parsing
                csv << part
              else
                raise MalformedCSVError, "Illegal quoting in line #{lineno + 1}."
              end
            end
          else
            # Regular ole unquoted field.
            csv << (part.empty? ? nil : part)
          end
        end

        # Replace tacked on @col_sep with @row_sep if we are still in an extended
        # column.
        csv[-1][-1] = @row_sep if in_extended_col

        if in_extended_col
          # if we're at eof?(), a quoted field wasn't closed...
          if @io.eof?
            raise MalformedCSVError,
                  "Unclosed quoted field on line #{lineno + 1}."
          elsif @field_size_limit and csv.last.sum(&:size) >= @field_size_limit
            raise MalformedCSVError, "Field size exceeded on line #{lineno + 1}."
          end
          # otherwise, we need to loop and pull some more data to complete the row
        else
          @lineno += 1

          # save fields unconverted fields, if needed...
          unconverted = csv.dup if @unconverted_fields

          # convert fields, if needed...
          csv = convert_fields(csv) unless @use_headers or @converters.empty?
          # parse out header rows and handle CSV::Row conversions...
          csv = parse_headers(csv)  if     @use_headers

          # inject unconverted fields and accessor, if requested...
          if @unconverted_fields and not csv.respond_to? :unconverted_fields
            add_unconverted_fields(csv, unconverted)
          end

          # return the results
          break csv
        end
      end
    end

Also aliased as: gets, readline
skip_blanks?() click to toggle source

Returns `true` blank lines are skipped by the parser. See `CSV::new` for details.

    # File lib/csv.rb, line 1643
    def skip_blanks?()        @skip_blanks        end

unconverted_fields?() click to toggle source

Returns `true` if unconverted_fields() to parsed results. See `CSV::new` for details.

    # File lib/csv.rb, line 1612
    def unconverted_fields?() @unconverted_fields end

write_headers?() click to toggle source

Returns `true` if headers are written in output. See `CSV::new` for details.

    # File lib/csv.rb, line 1627
    def write_headers?()      @write_headers      end

### Private Instance Methods

add_converter(var_name, const, name = nil, &converter) click to toggle source

The actual work method for adding converters, used by both `CSV.convert()` and `CSV.header_convert()`.

This method requires the `var_name` of the instance variable to place the converters in, the `const` `Hash` to lookup named converters in, and the normal parameters of the `CSV.convert()` and `CSV.header_convert()` methods.

    # File lib/csv.rb, line 2202
    def add_converter(var_name, const, name = nil, &converter)
      if name.nil?  # custom converter
        instance_variable_get("@#{var_name}") << converter
      else          # named converter
        combo = const[name]
        case combo
        when Array  # combo converter
          combo.each do |converter_name|
            add_converter(var_name, const, converter_name)
          end
        else        # individual named converter
          instance_variable_get("@#{var_name}") << combo
        end
      end
    end

add_unconverted_fields(row, fields) click to toggle source

This method injects an instance variable `unconverted_fields` into `row` and an accessor method for `row` called unconverted_fields(). The variable is set to the contents of `fields`.

    # File lib/csv.rb, line 2289
    def add_unconverted_fields(row, fields)
      class << row
        attr_reader :unconverted_fields
      end
      row.instance_eval { @unconverted_fields = fields }
      row
    end

convert_fields(fields, headers = false) click to toggle source

Processes `fields` with `@converters`, or `@header_converters` if `headers` is passed as `true`, returning the converted field set. Any converter that changes the field into something other than a String halts the pipeline of conversion for that field. This is primarily an efficiency shortcut.

    # File lib/csv.rb, line 2225
    def convert_fields(fields, headers = false)
      # see if we are converting headers or fields
      converters = headers ? @header_converters : @converters

      fields.map.with_index do |field, index|
        converters.each do |converter|
          break if field.nil?
          field = if converter.arity == 1  # straight field converter
            converter[field]
          else                             # FieldInfo converter
            header = @use_headers && !headers ? @headers[index] : nil
            converter[field, FieldInfo.new(index, lineno, header)]
          end
          break unless field.is_a? String  # short-circuit pipeline for speed
        end
        field  # final state of each field, converted or original
      end
    end

encode_re(*chunks) click to toggle source

Builds a regular expression in `@encoding`. All `chunks` will be transcoded to that encoding.

    # File lib/csv.rb, line 2312
    def encode_re(*chunks)
      Regexp.new(encode_str(*chunks))
    end

encode_str(*chunks) click to toggle source

Builds a String in `@encoding`. All `chunks` will be transcoded to that encoding.

    # File lib/csv.rb, line 2320
    def encode_str(*chunks)
      chunks.map { |chunk| chunk.encode(@encoding.name) }.join('')
    end

escape_re(str) click to toggle source

This method is an encoding safe version of `Regexp::escape()`. It will escape any characters that would change the meaning of a regular expression in the encoding of `str`. Regular expression characters that cannot be transcoded to the target encoding will be skipped and no escaping will be performed if a backslash cannot be transcoded.

    # File lib/csv.rb, line 2304
    def escape_re(str)
      str.gsub(@re_chars) {|c| @re_esc + c}
    end

init_comments(options) click to toggle source

Stores the pattern of comments to skip from the provided options.

The pattern must respond to `.match`, else `ArgumentError` is raised. Strings are converted to a `Regexp`.

See also `CSV.new`

    # File lib/csv.rb, line 2187
    def init_comments(options)
      @skip_lines = options.delete(:skip_lines)
      @skip_lines = Regexp.new(@skip_lines) if @skip_lines.is_a? String
      if @skip_lines and not @skip_lines.respond_to?(:match)
        raise ArgumentError, ":skip_lines has to respond to matches"
      end
    end

init_converters(options, field_name = :converters) click to toggle source

Loads any converters requested during construction.

If `field_name` is set `:converters` (the default) field converters are set. When `field_name` is `:header_converters` header converters are added instead.

The `:unconverted_fields` option is also activated for `:converters` calls, if requested.

    # File lib/csv.rb, line 2140
    def init_converters(options, field_name = :converters)
      if field_name == :converters
        @unconverted_fields = options.delete(:unconverted_fields)
      end

      instance_variable_set("@#{field_name}", Array.new)

      # find the correct method to add the converters
      convert = method(field_name.to_s.sub(/ers\Z/, ""))

      # load converters
      unless options[field_name].nil?
        # allow a single converter not wrapped in an Array
        unless options[field_name].is_a? Array
          options[field_name] = [options[field_name]]
        end
        # load each converter...
        options[field_name].each do |converter|
          if converter.is_a? Proc  # custom code block
            convert.call(&converter)
          else                     # by name
            convert.call(converter)
          end
        end
      end

      options.delete(field_name)
    end

init_headers(options) click to toggle source

Stores header row settings and loads header converters, if needed.

    # File lib/csv.rb, line 2170
    def init_headers(options)
      @use_headers    = options.delete(:headers)
      @return_headers = options.delete(:return_headers)
      @write_headers  = options.delete(:write_headers)

      # headers must be delayed until shift(), in case they need a row of content
      @headers = nil

      init_converters(options, :header_converters)
    end

init_parsers(options) click to toggle source

Pre-compiles parsers and stores them by name for access during reads.

    # File lib/csv.rb, line 2108
    def init_parsers(options)
      # store the parser behaviors
      @skip_blanks      = options.delete(:skip_blanks)
      @field_size_limit = options.delete(:field_size_limit)
      @liberal_parsing  = options.delete(:liberal_parsing)

      # prebuild Regexps for faster parsing
      esc_row_sep = escape_re(@row_sep)
      esc_quote   = escape_re(@quote_char)
      @parsers = {
        # for detecting parse errors
        quote_or_nl:    encode_re("[", esc_quote, "\r\n]"),
        nl_or_lf:       encode_re("[\r\n]"),
        stray_quote:    encode_re( "[^", esc_quote, "]", esc_quote,
                                   "[^", esc_quote, "]" ),
        # safer than chomp!()
        line_end:       encode_re(esc_row_sep, "\\z"),
        # illegal unquoted characters
        return_newline: encode_str("\r\n")
      }
    end

init_separators(options) click to toggle source

Stores the indicated separators for later use.

If auto-discovery was requested for `@row_sep`, this method will read ahead in the `@io` and try to find one. `ARGF`, `STDIN`, `STDOUT`, `STDERR` and any stream open for output only with a default `@row_sep` of `$INPUT_RECORD_SEPARATOR` (`$/`).

This method also establishes the quoting rules used for `CSV` output.

    # File lib/csv.rb, line 2009
    def init_separators(options)
      # store the selected separators
      @col_sep    = options.delete(:col_sep).to_s.encode(@encoding)
      @row_sep    = options.delete(:row_sep)  # encode after resolving :auto
      @quote_char = options.delete(:quote_char).to_s.encode(@encoding)

      if @quote_char.length != 1
        raise ArgumentError, ":quote_char has to be a single character String"
      end

      #
      # automatically discover row separator when requested
      # (not fully encoding safe)
      #
      if @row_sep == :auto
        if [ARGF, STDIN, STDOUT, STDERR].include?(@io) or
           (defined?(Zlib) and @io.class == Zlib::GzipWriter)
          @row_sep = $INPUT_RECORD_SEPARATOR
        else
          begin
            #
            # remember where we were (pos() will raise an exception if @io is pipe
            # or not opened for reading)
            #
            saved_pos = @io.pos
            while @row_sep == :auto
              #
              # if we run out of data, it's probably a single line
              # (ensure will set default value)
              #
              break unless sample = @io.gets(nil, 1024)
              # extend sample if we're unsure of the line ending
              if sample.end_with? encode_str("\r")
                sample << (@io.gets(nil, 1) || "")
              end

              # try to find a standard separator
              if sample =~ encode_re("\r\n?|\n")
                @row_sep = $&
                break
              end
            end

            # tricky seek() clone to work around GzipReader's lack of seek()
            @io.rewind
            # reset back to the remembered position
            while saved_pos > 1024  # avoid loading a lot of data into memory
              @io.read(1024)
              saved_pos -= 1024
            end
            @io.read(saved_pos) if saved_pos.nonzero?
          rescue IOError         # not opened for reading
            # do nothing:  ensure will set default
          rescue NoMethodError   # Zlib::GzipWriter doesn't have some IO methods
            # do nothing:  ensure will set default
          rescue SystemCallError # pipe
            # do nothing:  ensure will set default
          ensure
            #
            # set default if we failed to detect
            # (stream not opened for reading, a pipe, or a single line of data)
            #
            @row_sep = $INPUT_RECORD_SEPARATOR if @row_sep == :auto
          end
        end
      end
      @row_sep = @row_sep.to_s.encode(@encoding)

      # establish quoting rules
      @force_quotes   = options.delete(:force_quotes)
      do_quote        = lambda do |field|
        field         = String(field)
        encoded_quote = @quote_char.encode(field.encoding)
        encoded_quote                                +
        field.gsub(encoded_quote, encoded_quote * 2) +
        encoded_quote
      end
      quotable_chars = encode_str("\r\n", @col_sep, @quote_char)
      @quote         = if @force_quotes
        do_quote
      else
        lambda do |field|
          if field.nil?  # represent +nil+ fields as empty unquoted fields
            ""
          else
            field = String(field)  # Stringify fields
            # represent empty fields as empty quoted fields
            if field.empty? or
               field.count(quotable_chars).nonzero?
              do_quote.call(field)
            else
              field  # unquoted field
            end
          end
        end
      end
    end

parse_headers(row = nil) click to toggle source

This method is used to turn a finished `row` into a `CSV::Row`. Header rows are also dealt with here, either by returning a `CSV::Row` with identical headers and fields (save that the fields do not go through the converters) or by reading past them to return a field row. Headers are also saved in `@headers` for use in future rows.

When `nil`, `row` is assumed to be a header row not based on an actual row of the stream.

    # File lib/csv.rb, line 2254
    def parse_headers(row = nil)
      if @headers.nil?                # header row
        @headers = case @use_headers  # save headers
                   # Array of headers
                   when Array then @use_headers
                   # CSV header String
                   when String
                     self.class.parse_line( @use_headers,
                                            col_sep:    @col_sep,
                                            row_sep:    @row_sep,
                                            quote_char: @quote_char )
                   # first row is headers
                   else            row
                   end

        # prepare converted and unconverted copies
        row      = @headers                       if row.nil?
        @headers = convert_fields(@headers, true)
        @headers.each { |h| h.freeze if h.is_a? String }

        if @return_headers                                     # return headers
          return self.class::Row.new(@headers, row, true)
        elsif not [Array, String].include? @use_headers.class  # skip to field row
          return shift
        end
      end

      self.class::Row.new(@headers, convert_fields(row))  # field row
    end

raw_encoding(default = Encoding::ASCII_8BIT) click to toggle source

Returns the encoding of the internal `IO` object or the `default` if the encoding cannot be determined.

    # File lib/csv.rb, line 2330
    def raw_encoding(default = Encoding::ASCII_8BIT)
      if @io.respond_to? :internal_encoding
        @io.internal_encoding || @io.external_encoding
      elsif @io.is_a? StringIO
        @io.string.encoding
      elsif @io.respond_to? :encoding
        @io.encoding
      else
        default
      end
    end