Class CharStreams


  • public final class CharStreams
    extends java.lang.Object
    This class represents the primary interface for creating CharStreams from a variety of sources as of 4.7. The motivation was to support Unicode code points > U+FFFF. ANTLRInputStream and ANTLRFileStream are now deprecated in favor of the streams created by this interface. DEPRECATED: new ANTLRFileStream("myinputfile") NEW: CharStreams.fromFileName("myinputfile") WARNING: If you use both the deprecated and the new streams, you will see a nontrivial performance degradation. This speed hit is because the Lexer's internal code goes from a monomorphic to megamorphic dynamic dispatch to get characters from the input stream. Java's on-the-fly compiler (JIT) is unable to perform the same optimizations so stick with either the old or the new streams, if performance is a primary concern. See the extreme debugging and spelunking needed to identify this issue in our timing rig: https://github.com/antlr/antlr4/pull/1781 The ANTLR character streams still buffer all the input when you create the stream, as they have done for ~20 years. If you need unbuffered access, please note that it becomes challenging to create parse trees. The parse tree has to point to tokens which will either point into a stale location in an unbuffered stream or you have to copy the characters out of the buffer into the token. That defeats the purpose of unbuffered input. Per the ANTLR book, unbuffered streams are primarily useful for processing infinite streams *during the parse.* The new streams also use 8-bit buffers when possible so this new interface supports character streams that use half as much memory as the old ANTLRFileStream, which assumed 16-bit characters. A big shout out to Ben Hamilton (github bhamiltoncx) for his superhuman efforts across all targets to get true Unicode 3.1 support for U+10FFFF.
    Since:
    4.7
    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method Description
      static CharStream fromChannel​(java.nio.channels.ReadableByteChannel channel)
      Creates a CharStream given an opened ReadableByteChannel containing UTF-8 bytes.
      static CodePointCharStream fromChannel​(java.nio.channels.ReadableByteChannel channel, int bufferSize, java.nio.charset.CodingErrorAction decodingErrorAction, java.lang.String sourceName)
      Creates a CharStream given an opened ReadableByteChannel containing UTF-8 bytes.
      static CharStream fromChannel​(java.nio.channels.ReadableByteChannel channel, java.nio.charset.Charset charset)
      Creates a CharStream given an opened ReadableByteChannel and the charset of the bytes contained in the channel.
      static CodePointCharStream fromChannel​(java.nio.channels.ReadableByteChannel channel, java.nio.charset.Charset charset, int bufferSize, java.nio.charset.CodingErrorAction decodingErrorAction, java.lang.String sourceName, long inputSize)  
      static CharStream fromFileName​(java.lang.String fileName)
      Creates a CharStream given a string containing a path to a UTF-8 file on disk.
      static CharStream fromFileName​(java.lang.String fileName, java.nio.charset.Charset charset)
      Creates a CharStream given a string containing a path to a file on disk and the charset of the bytes contained in the file.
      static CharStream fromPath​(java.nio.file.Path path)
      Creates a CharStream given a path to a UTF-8 encoded file on disk.
      static CharStream fromPath​(java.nio.file.Path path, java.nio.charset.Charset charset)
      Creates a CharStream given a path to a file on disk and the charset of the bytes contained in the file.
      static CodePointCharStream fromReader​(java.io.Reader r)
      Creates a CharStream given a Reader.
      static CodePointCharStream fromReader​(java.io.Reader r, java.lang.String sourceName)
      Creates a CharStream given a Reader and its source name.
      static CharStream fromStream​(java.io.InputStream is)
      Creates a CharStream given an opened InputStream containing UTF-8 bytes.
      static CharStream fromStream​(java.io.InputStream is, java.nio.charset.Charset charset)
      Creates a CharStream given an opened InputStream and the charset of the bytes contained in the stream.
      static CharStream fromStream​(java.io.InputStream is, java.nio.charset.Charset charset, long inputSize)  
      static CodePointCharStream fromString​(java.lang.String s)
      Creates a CharStream given a String.
      static CodePointCharStream fromString​(java.lang.String s, java.lang.String sourceName)
      Creates a CharStream given a String and the sourceName from which it came.
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Method Detail

      • fromPath

        public static CharStream fromPath​(java.nio.file.Path path)
                                   throws java.io.IOException
        Creates a CharStream given a path to a UTF-8 encoded file on disk. Reads the entire contents of the file into the result before returning.
        Throws:
        java.io.IOException
      • fromPath

        public static CharStream fromPath​(java.nio.file.Path path,
                                          java.nio.charset.Charset charset)
                                   throws java.io.IOException
        Creates a CharStream given a path to a file on disk and the charset of the bytes contained in the file. Reads the entire contents of the file into the result before returning.
        Throws:
        java.io.IOException
      • fromFileName

        public static CharStream fromFileName​(java.lang.String fileName)
                                       throws java.io.IOException
        Creates a CharStream given a string containing a path to a UTF-8 file on disk. Reads the entire contents of the file into the result before returning.
        Throws:
        java.io.IOException
      • fromFileName

        public static CharStream fromFileName​(java.lang.String fileName,
                                              java.nio.charset.Charset charset)
                                       throws java.io.IOException
        Creates a CharStream given a string containing a path to a file on disk and the charset of the bytes contained in the file. Reads the entire contents of the file into the result before returning.
        Throws:
        java.io.IOException
      • fromStream

        public static CharStream fromStream​(java.io.InputStream is)
                                     throws java.io.IOException
        Creates a CharStream given an opened InputStream containing UTF-8 bytes. Reads the entire contents of the InputStream into the result before returning, then closes the InputStream.
        Throws:
        java.io.IOException
      • fromStream

        public static CharStream fromStream​(java.io.InputStream is,
                                            java.nio.charset.Charset charset)
                                     throws java.io.IOException
        Creates a CharStream given an opened InputStream and the charset of the bytes contained in the stream. Reads the entire contents of the InputStream into the result before returning, then closes the InputStream.
        Throws:
        java.io.IOException
      • fromStream

        public static CharStream fromStream​(java.io.InputStream is,
                                            java.nio.charset.Charset charset,
                                            long inputSize)
                                     throws java.io.IOException
        Throws:
        java.io.IOException
      • fromChannel

        public static CharStream fromChannel​(java.nio.channels.ReadableByteChannel channel)
                                      throws java.io.IOException
        Creates a CharStream given an opened ReadableByteChannel containing UTF-8 bytes. Reads the entire contents of the channel into the result before returning, then closes the channel.
        Throws:
        java.io.IOException
      • fromChannel

        public static CharStream fromChannel​(java.nio.channels.ReadableByteChannel channel,
                                             java.nio.charset.Charset charset)
                                      throws java.io.IOException
        Creates a CharStream given an opened ReadableByteChannel and the charset of the bytes contained in the channel. Reads the entire contents of the channel into the result before returning, then closes the channel.
        Throws:
        java.io.IOException
      • fromReader

        public static CodePointCharStream fromReader​(java.io.Reader r)
                                              throws java.io.IOException
        Creates a CharStream given a Reader. Closes the reader before returning.
        Throws:
        java.io.IOException
      • fromReader

        public static CodePointCharStream fromReader​(java.io.Reader r,
                                                     java.lang.String sourceName)
                                              throws java.io.IOException
        Creates a CharStream given a Reader and its source name. Closes the reader before returning.
        Throws:
        java.io.IOException
      • fromString

        public static CodePointCharStream fromString​(java.lang.String s,
                                                     java.lang.String sourceName)
        Creates a CharStream given a String and the sourceName from which it came.
      • fromChannel

        public static CodePointCharStream fromChannel​(java.nio.channels.ReadableByteChannel channel,
                                                      int bufferSize,
                                                      java.nio.charset.CodingErrorAction decodingErrorAction,
                                                      java.lang.String sourceName)
                                               throws java.io.IOException
        Creates a CharStream given an opened ReadableByteChannel containing UTF-8 bytes. Reads the entire contents of the channel into the result before returning, then closes the channel.
        Throws:
        java.io.IOException
      • fromChannel

        public static CodePointCharStream fromChannel​(java.nio.channels.ReadableByteChannel channel,
                                                      java.nio.charset.Charset charset,
                                                      int bufferSize,
                                                      java.nio.charset.CodingErrorAction decodingErrorAction,
                                                      java.lang.String sourceName,
                                                      long inputSize)
                                               throws java.io.IOException
        Throws:
        java.io.IOException