Package org.apache.solr.cli

Class SimplePostTool

java.lang.Object
org.apache.solr.cli.SimplePostTool
@Deprecated(since="9.2") public class SimplePostTool extends Object
Deprecated.
This class is being replaced with bin/solr tool. Please use PostTool instead.
A simple utility class for posting raw updates to a Solr server. It has a main method, so it can be run on the command line. View this not as a best-practice code example, but as a standalone example built with an explicit purpose of not having external jar dependencies.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static class 
    SimplePostTool.PageFetcherResult
    Deprecated.
    Utility class to hold the result form a page fetch

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final String
    DEFAULT_CONTENT_TYPE
    Deprecated.
     
    static final String
    DEFAULT_FILE_TYPES
    Deprecated.
     

  • Constructor Summary

    Constructors
    Constructor
    Description
    SimplePostTool()
    Deprecated.
     
    SimplePostTool(String mode, URI url, boolean auto, String type, String format, int recursive, int delay, String fileTypes, OutputStream out, boolean commit, boolean optimize, String[] args)
    Deprecated.
    Constructor which takes in all mandatory input for the tool to work.

  • Method Summary

    Modifier and Type
    Method
    Description
    static String
    appendParam(String url, String param)
    Deprecated.
    Appends a URL query parameter to a URL
    void
    commit()
    Deprecated.
    Does a simple commit operation
    protected String
    computeFullUrl(URL baseUrl, String link)
    Deprecated.
    Computes the full URL based on a base url and a possibly relative link found in the href param of an HTML anchor.
    void
    doGet(String url)
    Deprecated.
    Performs a simple get on the given URL
    void
    doGet(URL url)
    Deprecated.
    Performs a simple get on the given URL
    void
    execute()
    Deprecated.
    After initialization, call execute to start the post job.
    FileFilter
    getFileFilterFromFileTypes(String fileTypes)
    Deprecated.
     
    static NodeList
    getNodesFromXP(Node n, String xpath)
    Deprecated.
    Gets all nodes matching an XPath
    static String
    getXP(Node n, String xpath, boolean concatAll)
    Deprecated.
    Gets the string content of the matching an XPath
    protected static String
    guessType(File file)
    Deprecated.
    Guesses the type of file, based on file name suffix Returns "application/octet-stream" if no corresponding mimeMap type.
    protected static boolean
    isOn(String property)
    Deprecated.
    Tests if a string is either "true", "on", "yes" or "1"
    static void
    main(String[] args)
    Deprecated.
    See usage() for valid command line usage
    static Document
    makeDom(byte[] in)
    Deprecated.
    Takes a string as input and returns a DOM
    protected static String
    normalizeUrlEnding(String link)
    Deprecated.
    Normalizes a URL string by removing anchor part and trailing slash
    void
    optimize()
    Deprecated.
    Does a simple optimize operation
    protected static SimplePostTool
    parseArgsAndInit(String[] args)
    Deprecated.
    Parses incoming arguments and system params and initializes the tool
    boolean
    postData(InputStream data, Long length, OutputStream output, String type, URI uri)
    Deprecated.
    Reads data from the data stream and posts it to solr, writes to the response to output
    void
    postFile(File file, OutputStream output, String type)
    Deprecated.
    Opens the file and posts its contents to the solrUrl, writes to response to output.
    int
    postFiles(File[] files, int startIndexInArgs, OutputStream out, String type)
    Deprecated.
    Post all filenames provided in args
    int
    postFiles(String[] args, int startIndexInArgs, OutputStream out, String type)
    Deprecated.
    Post all filenames provided in args
    int
    postWebPages(String[] args, int startIndexInArgs, OutputStream out)
    Deprecated.
    This method takes as input a list of start URL strings for crawling, converts the URL strings to URI strings and adds each one to a backlog and then starts crawling
    static InputStream
    stringToStream(String s)
    Deprecated.
    Converts a string to an input stream
    protected boolean
    typeSupported(String type)
    Deprecated.
    Uses the mime-type map to reverse lookup whether the file ending for our type is supported by the fileTypes option
    protected int
    webCrawl(int level, OutputStream out)
    Deprecated.
    A very simple crawler, pulling URLs to fetch from a backlog and then recurses N levels deep if recursive>0.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

  • Constructor Details

    • SimplePostTool

      public SimplePostTool(String mode, URI url, boolean auto, String type, String format, int recursive, int delay, String fileTypes, OutputStream out, boolean commit, boolean optimize, String[] args)
      Deprecated.
      Constructor which takes in all mandatory input for the tool to work. Also see usage() for further explanation of the params.
      Parameters:
      mode - whether to post files, web pages, params or stdin
      url - the Solr base Url to post to, should end with /update
      auto - if true, we'll guess type and add resourcename/url
      type - content-type of the data you are posting
      recursive - number of levels for file/web mode, or 0 if one file only
      delay - if recursive then delay will be the wait time between posts
      fileTypes - a comma separated list of file-name endings to accept for file/web
      out - an OutputStream to write output to, e.g. stdout to print to console
      commit - if true, will commit at end of posting
      optimize - if true, will optimize at end of posting
      args - a String[] of arguments, varies between modes

    • SimplePostTool

      public SimplePostTool()
      Deprecated.

  • Method Details

    • main

      public static void main(String[] args)
      Deprecated.
      See usage() for valid command line usage
      Parameters:
      args - the params on the command line

    • execute

      public void execute()
      Deprecated.
      After initialization, call execute to start the post job. This method delegates to the correct mode method.

    • parseArgsAndInit

      protected static SimplePostTool parseArgsAndInit(String[] args)
      Deprecated.
      Parses incoming arguments and system params and initializes the tool
      Parameters:
      args - the incoming cmd line args
      Returns:
      an instance of SimplePostTool

    • postFiles

      public int postFiles(String[] args, int startIndexInArgs, OutputStream out, String type)
      Deprecated.
      Post all filenames provided in args
      Parameters:
      args - array of file names
      startIndexInArgs - offset to start
      out - output stream to post data to
      type - default content-type to use when posting (may be overridden in auto mode)
      Returns:
      number of files posted

    • postFiles

      public int postFiles(File[] files, int startIndexInArgs, OutputStream out, String type)
      Deprecated.
      Post all filenames provided in args
      Parameters:
      files - array of Files
      startIndexInArgs - offset to start
      out - output stream to post data to
      type - default content-type to use when posting (may be overridden in auto mode)
      Returns:
      number of files posted

    • postWebPages

      public int postWebPages(String[] args, int startIndexInArgs, OutputStream out)
      Deprecated.
      This method takes as input a list of start URL strings for crawling, converts the URL strings to URI strings and adds each one to a backlog and then starts crawling
      Parameters:
      args - the raw input args from main()
      startIndexInArgs - offset for where to start
      out - outputStream to write results to
      Returns:
      the number of web pages posted

    • normalizeUrlEnding

      protected static String normalizeUrlEnding(String link)
      Deprecated.
      Normalizes a URL string by removing anchor part and trailing slash
      Returns:
      the normalized URL string

    • webCrawl

      protected int webCrawl(int level, OutputStream out)
      Deprecated.
      A very simple crawler, pulling URLs to fetch from a backlog and then recurses N levels deep if recursive>0. Links are parsed from HTML through first getting an XHTML version using SolrCell with extractOnly, and followed if they are local. The crawler pauses for a default delay of 10 seconds between each fetch, this can be configured in the delay variable. This is only meant for test purposes, as it does not respect robots or anything else fancy :)
      Parameters:
      level - which level to crawl
      out - output stream to write to
      Returns:
      number of pages crawled on this level and below

    • computeFullUrl

      protected String computeFullUrl(URL baseUrl, String link) throws MalformedURLException, URISyntaxException
      Deprecated.
      Computes the full URL based on a base url and a possibly relative link found in the href param of an HTML anchor.
      Parameters:
      baseUrl - the base url from where the link was found
      link - the absolute or relative link
      Returns:
      the string version of the full URL
      Throws:
      MalformedURLException
      URISyntaxException

    • typeSupported

      protected boolean typeSupported(String type)
      Deprecated.
      Uses the mime-type map to reverse lookup whether the file ending for our type is supported by the fileTypes option
      Parameters:
      type - what content-type to lookup
      Returns:
      true if this is a supported content type

    • isOn

      protected static boolean isOn(String property)
      Deprecated.
      Tests if a string is either "true", "on", "yes" or "1"
      Parameters:
      property - the string to test
      Returns:
      true if "on"

    • commit

      public void commit()
      Deprecated.
      Does a simple commit operation

    • optimize

      public void optimize()
      Deprecated.
      Does a simple optimize operation

    • appendParam

      public static String appendParam(String url, String param)
      Deprecated.
      Appends a URL query parameter to a URL
      Parameters:
      url - the original URL
      param - the parameter(s) to append, separated by "&"
      Returns:
      the string version of the resulting URL

    • postFile

      public void postFile(File file, OutputStream output, String type)
      Deprecated.
      Opens the file and posts its contents to the solrUrl, writes to response to output.

    • guessType

      protected static String guessType(File file)
      Deprecated.
      Guesses the type of file, based on file name suffix Returns "application/octet-stream" if no corresponding mimeMap type.
      Parameters:
      file - the file
      Returns:
      the content-type guessed

    • doGet

      public void doGet(String url)
      Deprecated.
      Performs a simple get on the given URL

    • doGet

      public void doGet(URL url)
      Deprecated.
      Performs a simple get on the given URL

    • postData

      public boolean postData(InputStream data, Long length, OutputStream output, String type, URI uri)
      Deprecated.
      Reads data from the data stream and posts it to solr, writes to the response to output
      Returns:
      true if success

    • stringToStream

      public static InputStream stringToStream(String s)
      Deprecated.
      Converts a string to an input stream
      Parameters:
      s - the string
      Returns:
      the input stream

    • getFileFilterFromFileTypes

      public FileFilter getFileFilterFromFileTypes(String fileTypes)
      Deprecated.

    • getNodesFromXP

      public static NodeList getNodesFromXP(Node n, String xpath) throws XPathExpressionException
      Deprecated.
      Gets all nodes matching an XPath
      Throws:
      XPathExpressionException

    • getXP

      public static String getXP(Node n, String xpath, boolean concatAll) throws XPathExpressionException
      Deprecated.
      Gets the string content of the matching an XPath
      Parameters:
      n - the node (or doc)
      xpath - the xpath string
      concatAll - if true, text from all matching nodes will be concatenated, else only the first returned
      Throws:
      XPathExpressionException

    • makeDom

      public static Document makeDom(byte[] in) throws SAXException, IOException, ParserConfigurationException
      Deprecated.
      Takes a string as input and returns a DOM
      Throws:
      SAXException
      IOException
      ParserConfigurationException