Package org.apache.solr.cli
Class SimplePostTool
- java.lang.Object
-
- org.apache.solr.cli.SimplePostTool
-
@Deprecated(since="9.2") public class SimplePostTool extends Object
Deprecated.This class is being replaced with bin/solr tool. Please usePostTool
instead.A simple utility class for posting raw updates to a Solr server. It has a main method, so it can be run on the command line. View this not as a best-practice code example, but as a standalone example built with an explicit purpose of not having external jar dependencies.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
SimplePostTool.PageFetcherResult
Deprecated.Utility class to hold the result form a page fetch
-
Field Summary
Fields Modifier and Type Field Description static String
DEFAULT_CONTENT_TYPE
Deprecated.static String
DEFAULT_FILE_TYPES
Deprecated.
-
Constructor Summary
Constructors Constructor Description SimplePostTool()
Deprecated.SimplePostTool(String mode, URL url, boolean auto, String type, String format, int recursive, int delay, String fileTypes, OutputStream out, boolean commit, boolean optimize, String[] args)
Deprecated.Constructor which takes in all mandatory input for the tool to work.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Deprecated Methods Modifier and Type Method Description static String
appendParam(String url, String param)
Deprecated.Appends a URL query parameter to a URLprotected static URL
appendUrlPath(URL url, String append)
Deprecated.Appends to the path of the URLvoid
commit()
Deprecated.Does a simple commit operationprotected String
computeFullUrl(URL baseUrl, String link)
Deprecated.Computes the full URL based on a base url and a possibly relative link found in the href param of an HTML anchor.void
doGet(String url)
Deprecated.Performs a simple get on the given URLvoid
doGet(URL url)
Deprecated.Performs a simple get on the given URLvoid
execute()
Deprecated.After initialization, call execute to start the post job.FileFilter
getFileFilterFromFileTypes(String fileTypes)
Deprecated.static NodeList
getNodesFromXP(Node n, String xpath)
Deprecated.Gets all nodes matching an XPathstatic String
getXP(Node n, String xpath, boolean concatAll)
Deprecated.Gets the string content of the matching an XPathprotected static String
guessType(File file)
Deprecated.Guesses the type of file, based on file name suffix Returns "application/octet-stream" if no corresponding mimeMap type.protected static boolean
isOn(String property)
Deprecated.Tests if a string is either "true", "on", "yes" or "1"static void
main(String[] args)
Deprecated.See usage() for valid command line usagestatic Document
makeDom(byte[] in)
Deprecated.Takes a string as input and returns a DOMprotected static String
normalizeUrlEnding(String link)
Deprecated.Normalizes a URL string by removing anchor part and trailing slashvoid
optimize()
Deprecated.Does a simple optimize operationprotected static SimplePostTool
parseArgsAndInit(String[] args)
Deprecated.Parses incoming arguments and system params and initializes the toolboolean
postData(InputStream data, Long length, OutputStream output, String type, URL url)
Deprecated.Reads data from the data stream and posts it to solr, writes to the response to outputvoid
postFile(File file, OutputStream output, String type)
Deprecated.Opens the file and posts its contents to the solrUrl, writes to response to output.int
postFiles(File[] files, int startIndexInArgs, OutputStream out, String type)
Deprecated.Post all filenames provided in argsint
postFiles(String[] args, int startIndexInArgs, OutputStream out, String type)
Deprecated.Post all filenames provided in argsint
postWebPages(String[] args, int startIndexInArgs, OutputStream out)
Deprecated.This method takes as input a list of start URL strings for crawling, converts the URL strings to URI strings and adds each one to a backlog and then starts crawlingstatic InputStream
stringToStream(String s)
Deprecated.Converts a string to an input streamprotected boolean
typeSupported(String type)
Deprecated.Uses the mime-type map to reverse lookup whether the file ending for our type is supported by the fileTypes optionprotected int
webCrawl(int level, OutputStream out)
Deprecated.A very simple crawler, pulling URLs to fetch from a backlog and then recurses N levels deep if recursive>0.
-
-
-
Field Detail
-
DEFAULT_CONTENT_TYPE
public static final String DEFAULT_CONTENT_TYPE
Deprecated.- See Also:
- Constant Field Values
-
DEFAULT_FILE_TYPES
public static final String DEFAULT_FILE_TYPES
Deprecated.- See Also:
- Constant Field Values
-
-
Constructor Detail
-
SimplePostTool
public SimplePostTool(String mode, URL url, boolean auto, String type, String format, int recursive, int delay, String fileTypes, OutputStream out, boolean commit, boolean optimize, String[] args)
Deprecated.Constructor which takes in all mandatory input for the tool to work. Also see usage() for further explanation of the params.- Parameters:
mode
- whether to post files, web pages, params or stdinurl
- the Solr base Url to post to, should end with /updateauto
- if true, we'll guess type and add resourcename/urltype
- content-type of the data you are postingrecursive
- number of levels for file/web mode, or 0 if one file onlydelay
- if recursive then delay will be the wait time between postsfileTypes
- a comma separated list of file-name endings to accept for file/webout
- an OutputStream to write output to, e.g. stdout to print to consolecommit
- if true, will commit at end of postingoptimize
- if true, will optimize at end of postingargs
- a String[] of arguments, varies between modes
-
SimplePostTool
public SimplePostTool()
Deprecated.
-
-
Method Detail
-
main
public static void main(String[] args)
Deprecated.See usage() for valid command line usage- Parameters:
args
- the params on the command line
-
execute
public void execute()
Deprecated.After initialization, call execute to start the post job. This method delegates to the correct mode method.
-
parseArgsAndInit
protected static SimplePostTool parseArgsAndInit(String[] args)
Deprecated.Parses incoming arguments and system params and initializes the tool- Parameters:
args
- the incoming cmd line args- Returns:
- an instance of SimplePostTool
-
postFiles
public int postFiles(String[] args, int startIndexInArgs, OutputStream out, String type)
Deprecated.Post all filenames provided in args- Parameters:
args
- array of file namesstartIndexInArgs
- offset to startout
- output stream to post data totype
- default content-type to use when posting (may be overridden in auto mode)- Returns:
- number of files posted
-
postFiles
public int postFiles(File[] files, int startIndexInArgs, OutputStream out, String type)
Deprecated.Post all filenames provided in args- Parameters:
files
- array of FilesstartIndexInArgs
- offset to startout
- output stream to post data totype
- default content-type to use when posting (may be overridden in auto mode)- Returns:
- number of files posted
-
postWebPages
public int postWebPages(String[] args, int startIndexInArgs, OutputStream out)
Deprecated.This method takes as input a list of start URL strings for crawling, converts the URL strings to URI strings and adds each one to a backlog and then starts crawling- Parameters:
args
- the raw input args from main()startIndexInArgs
- offset for where to startout
- outputStream to write results to- Returns:
- the number of web pages posted
-
normalizeUrlEnding
protected static String normalizeUrlEnding(String link)
Deprecated.Normalizes a URL string by removing anchor part and trailing slash- Returns:
- the normalized URL string
-
webCrawl
protected int webCrawl(int level, OutputStream out)
Deprecated.A very simple crawler, pulling URLs to fetch from a backlog and then recurses N levels deep if recursive>0. Links are parsed from HTML through first getting an XHTML version using SolrCell with extractOnly, and followed if they are local. The crawler pauses for a default delay of 10 seconds between each fetch, this can be configured in the delay variable. This is only meant for test purposes, as it does not respect robots or anything else fancy :)- Parameters:
level
- which level to crawlout
- output stream to write to- Returns:
- number of pages crawled on this level and below
-
computeFullUrl
protected String computeFullUrl(URL baseUrl, String link) throws MalformedURLException, URISyntaxException
Deprecated.Computes the full URL based on a base url and a possibly relative link found in the href param of an HTML anchor.- Parameters:
baseUrl
- the base url from where the link was foundlink
- the absolute or relative link- Returns:
- the string version of the full URL
- Throws:
MalformedURLException
URISyntaxException
-
typeSupported
protected boolean typeSupported(String type)
Deprecated.Uses the mime-type map to reverse lookup whether the file ending for our type is supported by the fileTypes option- Parameters:
type
- what content-type to lookup- Returns:
- true if this is a supported content type
-
isOn
protected static boolean isOn(String property)
Deprecated.Tests if a string is either "true", "on", "yes" or "1"- Parameters:
property
- the string to test- Returns:
- true if "on"
-
commit
public void commit()
Deprecated.Does a simple commit operation
-
optimize
public void optimize()
Deprecated.Does a simple optimize operation
-
appendParam
public static String appendParam(String url, String param)
Deprecated.Appends a URL query parameter to a URL- Parameters:
url
- the original URLparam
- the parameter(s) to append, separated by "&"- Returns:
- the string version of the resulting URL
-
postFile
public void postFile(File file, OutputStream output, String type)
Deprecated.Opens the file and posts its contents to the solrUrl, writes to response to output.
-
appendUrlPath
protected static URL appendUrlPath(URL url, String append) throws MalformedURLException, URISyntaxException
Deprecated.Appends to the path of the URL- Parameters:
url
- the URLappend
- the path to append- Returns:
- the final URL version
- Throws:
MalformedURLException
URISyntaxException
-
guessType
protected static String guessType(File file)
Deprecated.Guesses the type of file, based on file name suffix Returns "application/octet-stream" if no corresponding mimeMap type.- Parameters:
file
- the file- Returns:
- the content-type guessed
-
doGet
public void doGet(String url)
Deprecated.Performs a simple get on the given URL
-
doGet
public void doGet(URL url)
Deprecated.Performs a simple get on the given URL
-
postData
public boolean postData(InputStream data, Long length, OutputStream output, String type, URL url)
Deprecated.Reads data from the data stream and posts it to solr, writes to the response to output- Returns:
- true if success
-
stringToStream
public static InputStream stringToStream(String s)
Deprecated.Converts a string to an input stream- Parameters:
s
- the string- Returns:
- the input stream
-
getFileFilterFromFileTypes
public FileFilter getFileFilterFromFileTypes(String fileTypes)
Deprecated.
-
getNodesFromXP
public static NodeList getNodesFromXP(Node n, String xpath) throws XPathExpressionException
Deprecated.Gets all nodes matching an XPath- Throws:
XPathExpressionException
-
getXP
public static String getXP(Node n, String xpath, boolean concatAll) throws XPathExpressionException
Deprecated.Gets the string content of the matching an XPath- Parameters:
n
- the node (or doc)xpath
- the xpath stringconcatAll
- if true, text from all matching nodes will be concatenated, else only the first returned- Throws:
XPathExpressionException
-
makeDom
public static Document makeDom(byte[] in) throws SAXException, IOException, ParserConfigurationException
Deprecated.Takes a string as input and returns a DOM
-
-