visone manual - User contributions [en]

NLP tab

2011-07-14T19:49:00Z

Fratz: /* word net analysis */

The NLP tab contains the visone Natural Language Processing functionality which can be used to create networks from text written in a natural language (usually English).

The NLP tab is not shown by default. In order for it to be enabled, visone must be started with the command line option
-e de.visone.nlp.NLPExtension
If texts with long sentences are to be processed, it is furthermore desirable to add <code>-Xmx1g</code> to prevent visone from running out of memory during network creation.

In addition, the files <code>englishPCFG.ser.gz</code> and <code>left3words-wsj-0-18.tagger</code> should be downloaded from the [http://nlp.stanford.edu/ Stanford Natural Language Processing Group's website] and placed in the visone working directory. This will allow ''centering resonance analysis'' and ''word net analysis'' to work without explicitly specifying a parser or tagger file (see below).

== centering resonance analysis ==
This performs a [[CRA]] analysis of the input text.
* '''quick layout''' does a quick layout after network creation, which may take some time for huge networks.
* The '''stopword filter''' can either use the builtin list of stopwords (common function words that can be assumed to carry no meaning for the topic of the text), or a file containing one lowercase stopword per line, with empty lines and comments (starting with <code>#</code>) ignored. Also, a minimum word length can be set, so that all words shorter than this length are always ignored.
* A comma-separated list of '''center tags''' can be specified. These are the node names as used by the parser, eg. NP (noun phrase) or VP (verb phrase). Phrases can be nested, such that an NP contains more NPs. If '''take centers from bottom level''' is selected, the inner ones are used, else the top one, thus producing a much denser network.
* The '''parser file''' must be specified, however the default path points to <code>englishPCFG.ser.gz</code> in the visone working directory, so if the file is placed there as recommended, nothing should need to be selected.

== word net analysis ==
Performs a [[WNA]] analysis of the input text.
* For WNA, either a '''window size''' must be set, or if '''use co-occurence''' is selected, all words within a sentence are connected.
* WNA can filter according to POS (part-of-speech) tags. The default tag set selects nouns, verbs, adjectives, adverbs and foreign words. Alternatively, an external file with one tag per line can be specified; in this case, empty lines and comments (starting with <code>#</code>) are ignored. The default tagger uses the [ftp://ftp.cis.upenn.edu/pub/treebank/doc/tagguide.ps.gz Penn Treebank Tagset].
* The '''tagger file''' must be specified, however the default path points to <code>left3words-wsj-0-18.tagger</code> in the visone working directory, so if the file is placed there as recommended, nothing should need to be selected.

See the [[#centering resonance analysis|centering resonance analysis section]] for the '''quick layout''' and '''stopword filter''' options.

== crawl link structure ==
This function captures the link structure of a website into a network. One node is created for each page, and a (directed) edge for every link from one page to another. Each node will have the following attributes:
* '''url''': the URL of the page
* '''text''': the text content of the page (if enabled)
Each edge will have an attribute named '''linktext''' containing the text of the hyperlink represented by that edge. In addition, if a link was found but not followed, it will be marked as unconfirmed.

All that is required to use the crawler is the '''url''' field, which must be filled with the URL of the page at which the crawler should start, such as <code><nowiki>http://visone.info/wiki/index.php/Main_Page</nowiki></code> for the visone wiki. After crawling has finished, the network does ''not'' get layouted to allow handling of extremely large graphs, thus all nodes will be show lying on a single point. Click the quick layout button to see the network structure.

For finer control, the following options can be adjusted:
* '''browser''': The browser that visone will masquerade as, to avoid being blocked. Select one from the list, or paste any user agent string.
* '''max depth''': The maximum length of a chain of links that will be followed. For example, 0 will not follow any links, 1 will follow all links on the start page, and 2 will follow all links on the start page and on pages referenced by the start page.
* '''page limit''': The maximum number of pages that will be ''downloaded''. This is ''not'' a limit on the number of nodes created, because nodes are created for every page that is found, even if it is never downloaded.
* '''same host only''': If selected, links to different hosts will not be followed. For example, if a page on <code>visone.info</code> links to another page on <code>visone.info</code>, this link will be followed, but a link to <code>google.com</code> will not.
* '''store text''': If selected, the text content of every page that is downloaded will be stored in the '''text''' attribute of its node.

=== crawler expert options ===
* '''crawl rule selection''': This option decides which parts of the webpage will be considered textual content.
** '''standard''' uses a default rule that should work for most websites.
** '''domain specific''' selects the rule based on the URL, so that for example only the actual text of Wikipedia articles is extracted. The name of the rule file is generated from the URL, for example, the rule file for <code><nowiki>http://www.informatik.uni-konstanz.de/algo</nowiki></code> is <code>algo.www.informatik.uni-konstanz.de.rule</code>. Some rules, such as the one for Wikipedia, are already built in.
** '''specified''' uses the rule selected in the '''crawl rule''' dropdown.
* '''cookies''': If the website requires a login, paste the HTTP <code>Cookie:</code> header value here. (Obtaining this value might be complicated, though.)
* '''ignore errors''': If this option is ''not'' selected, any error encountered during crawling, such as the common ''page not found'', will terminate the crawler. Unchecking is only recommended if there are known to be no dead links.
* '''mean delay''': Delay after a successful page download, in milliseconds.
* '''cool down after error''': Delay after a failed page download, in milliseconds.
* '''respect nofollow''': If selected, visone will observe the [http://en.wikipedia.org/wiki/Nofollow nofollow] specification, and not follow any links such marked.

== crawl text ==
The purpose of this function is to download the textual content of a singe webpage and store it in a file that can be used as an input file for the text analysis methods discussed above.

Use '''output file''' to select the name of the file to (over-)write. For the '''url''', '''browser''' and expert options, see [[#crawl link structure|above]].

NLP tab

2011-07-14T19:32:29Z

Fratz: /* centering resonance analysis */ typo

The NLP tab contains the visone Natural Language Processing functionality which can be used to create networks from text written in a natural language (usually English).

The NLP tab is not shown by default. In order for it to be enabled, visone must be started with the command line option
-e de.visone.nlp.NLPExtension
If texts with long sentences are to be processed, it is furthermore desirable to add <code>-Xmx1g</code> to prevent visone from running out of memory during network creation.

In addition, the files <code>englishPCFG.ser.gz</code> and <code>left3words-wsj-0-18.tagger</code> should be downloaded from the [http://nlp.stanford.edu/ Stanford Natural Language Processing Group's website] and placed in the visone working directory. This will allow ''centering resonance analysis'' and ''word net analysis'' to work without explicitly specifying a parser or tagger file (see below).

== centering resonance analysis ==
This performs a [[CRA]] analysis of the input text.
* '''quick layout''' does a quick layout after network creation, which may take some time for huge networks.
* The '''stopword filter''' can either use the builtin list of stopwords (common function words that can be assumed to carry no meaning for the topic of the text), or a file containing one lowercase stopword per line, with empty lines and comments (starting with <code>#</code>) ignored. Also, a minimum word length can be set, so that all words shorter than this length are always ignored.
* A comma-separated list of '''center tags''' can be specified. These are the node names as used by the parser, eg. NP (noun phrase) or VP (verb phrase). Phrases can be nested, such that an NP contains more NPs. If '''take centers from bottom level''' is selected, the inner ones are used, else the top one, thus producing a much denser network.
* The '''parser file''' must be specified, however the default path points to <code>englishPCFG.ser.gz</code> in the visone working directory, so if the file is placed there as recommended, nothing should need to be selected.

== word net analysis ==
...

== crawl link structure ==
This function captures the link structure of a website into a network. One node is created for each page, and a (directed) edge for every link from one page to another. Each node will have the following attributes:
* '''url''': the URL of the page
* '''text''': the text content of the page (if enabled)
Each edge will have an attribute named '''linktext''' containing the text of the hyperlink represented by that edge. In addition, if a link was found but not followed, it will be marked as unconfirmed.

All that is required to use the crawler is the '''url''' field, which must be filled with the URL of the page at which the crawler should start, such as <code><nowiki>http://visone.info/wiki/index.php/Main_Page</nowiki></code> for the visone wiki. After crawling has finished, the network does ''not'' get layouted to allow handling of extremely large graphs, thus all nodes will be show lying on a single point. Click the quick layout button to see the network structure.

For finer control, the following options can be adjusted:
* '''browser''': The browser that visone will masquerade as, to avoid being blocked. Select one from the list, or paste any user agent string.
* '''max depth''': The maximum length of a chain of links that will be followed. For example, 0 will not follow any links, 1 will follow all links on the start page, and 2 will follow all links on the start page and on pages referenced by the start page.
* '''page limit''': The maximum number of pages that will be ''downloaded''. This is ''not'' a limit on the number of nodes created, because nodes are created for every page that is found, even if it is never downloaded.
* '''same host only''': If selected, links to different hosts will not be followed. For example, if a page on <code>visone.info</code> links to another page on <code>visone.info</code>, this link will be followed, but a link to <code>google.com</code> will not.
* '''store text''': If selected, the text content of every page that is downloaded will be stored in the '''text''' attribute of its node.

=== crawler expert options ===
* '''crawl rule selection''': This option decides which parts of the webpage will be considered textual content.
** '''standard''' uses a default rule that should work for most websites.
** '''domain specific''' selects the rule based on the URL, so that for example only the actual text of Wikipedia articles is extracted. The name of the rule file is generated from the URL, for example, the rule file for <code><nowiki>http://www.informatik.uni-konstanz.de/algo</nowiki></code> is <code>algo.www.informatik.uni-konstanz.de.rule</code>. Some rules, such as the one for Wikipedia, are already built in.
** '''specified''' uses the rule selected in the '''crawl rule''' dropdown.
* '''cookies''': If the website requires a login, paste the HTTP <code>Cookie:</code> header value here. (Obtaining this value might be complicated, though.)
* '''ignore errors''': If this option is ''not'' selected, any error encountered during crawling, such as the common ''page not found'', will terminate the crawler. Unchecking is only recommended if there are known to be no dead links.
* '''mean delay''': Delay after a successful page download, in milliseconds.
* '''cool down after error''': Delay after a failed page download, in milliseconds.
* '''respect nofollow''': If selected, visone will observe the [http://en.wikipedia.org/wiki/Nofollow nofollow] specification, and not follow any links such marked.

== crawl text ==
The purpose of this function is to download the textual content of a singe webpage and store it in a file that can be used as an input file for the text analysis methods discussed above.

Use '''output file''' to select the name of the file to (over-)write. For the '''url''', '''browser''' and expert options, see [[#crawl link structure|above]].

NLP tab

2011-07-14T19:30:24Z

Fratz: /* centering resonance analysis */ fix link

The NLP tab contains the visone Natural Language Processing functionality which can be used to create networks from text written in a natural language (usually English).

The NLP tab is not shown by default. In order for it to be enabled, visone must be started with the command line option
-e de.visone.nlp.NLPExtension
If texts with long sentences are to be processed, it is furthermore desirable to add <code>-Xmx1g</code> to prevent visone from running out of memory during network creation.

In addition, the files <code>englishPCFG.ser.gz</code> and <code>left3words-wsj-0-18.tagger</code> should be downloaded from the [http://nlp.stanford.edu/ Stanford Natural Language Processing Group's website] and placed in the visone working directory. This will allow ''centering resonance analysis'' and ''word net analysis'' to work without explicitly specifying a parser or tagger file (see below).

== centering resonance analysis ==
This performs a [[CRA]] analysis of the input text.
* '''quick layout''' does a quick layout after network creation, which may take some time for huge networks.
* The '''stopwords filter''' can either use the builtin list of stopwords (common function words that can be assumed to carry no meaning for the topic of the text), or a file containing one lowercase stopword per line, with empty lines and comments (starting with <code>#</code>) ignored. Also, a minimum word length can be set, so that all words shorter than this length are always ignored.
* A comma-separated list of '''center tags''' can be specified. These are the node names as used by the parser, eg. NP (noun phrase) or VP (verb phrase). Phrases can be nested, such that an NP contains more NPs. If '''take centers from bottom level''' is selected, the inner ones are used, else the top one, thus producing a much denser network.
* The '''parser file''' must be specified, however the default path points to <code>englishPCFG.ser.gz</code> in the visone working directory, so if the file is placed there as recommended, nothing should need to be selected.

== word net analysis ==
...

== crawl link structure ==
This function captures the link structure of a website into a network. One node is created for each page, and a (directed) edge for every link from one page to another. Each node will have the following attributes:
* '''url''': the URL of the page
* '''text''': the text content of the page (if enabled)
Each edge will have an attribute named '''linktext''' containing the text of the hyperlink represented by that edge. In addition, if a link was found but not followed, it will be marked as unconfirmed.

All that is required to use the crawler is the '''url''' field, which must be filled with the URL of the page at which the crawler should start, such as <code><nowiki>http://visone.info/wiki/index.php/Main_Page</nowiki></code> for the visone wiki. After crawling has finished, the network does ''not'' get layouted to allow handling of extremely large graphs, thus all nodes will be show lying on a single point. Click the quick layout button to see the network structure.

For finer control, the following options can be adjusted:
* '''browser''': The browser that visone will masquerade as, to avoid being blocked. Select one from the list, or paste any user agent string.
* '''max depth''': The maximum length of a chain of links that will be followed. For example, 0 will not follow any links, 1 will follow all links on the start page, and 2 will follow all links on the start page and on pages referenced by the start page.
* '''page limit''': The maximum number of pages that will be ''downloaded''. This is ''not'' a limit on the number of nodes created, because nodes are created for every page that is found, even if it is never downloaded.
* '''same host only''': If selected, links to different hosts will not be followed. For example, if a page on <code>visone.info</code> links to another page on <code>visone.info</code>, this link will be followed, but a link to <code>google.com</code> will not.
* '''store text''': If selected, the text content of every page that is downloaded will be stored in the '''text''' attribute of its node.

=== crawler expert options ===
* '''crawl rule selection''': This option decides which parts of the webpage will be considered textual content.
** '''standard''' uses a default rule that should work for most websites.
** '''domain specific''' selects the rule based on the URL, so that for example only the actual text of Wikipedia articles is extracted. The name of the rule file is generated from the URL, for example, the rule file for <code><nowiki>http://www.informatik.uni-konstanz.de/algo</nowiki></code> is <code>algo.www.informatik.uni-konstanz.de.rule</code>. Some rules, such as the one for Wikipedia, are already built in.
** '''specified''' uses the rule selected in the '''crawl rule''' dropdown.
* '''cookies''': If the website requires a login, paste the HTTP <code>Cookie:</code> header value here. (Obtaining this value might be complicated, though.)
* '''ignore errors''': If this option is ''not'' selected, any error encountered during crawling, such as the common ''page not found'', will terminate the crawler. Unchecking is only recommended if there are known to be no dead links.
* '''mean delay''': Delay after a successful page download, in milliseconds.
* '''cool down after error''': Delay after a failed page download, in milliseconds.
* '''respect nofollow''': If selected, visone will observe the [http://en.wikipedia.org/wiki/Nofollow nofollow] specification, and not follow any links such marked.

== crawl text ==
The purpose of this function is to download the textual content of a singe webpage and store it in a file that can be used as an input file for the text analysis methods discussed above.

Use '''output file''' to select the name of the file to (over-)write. For the '''url''', '''browser''' and expert options, see [[#crawl link structure|above]].

NLP tab

2011-07-14T19:30:02Z

Fratz: /* centering resonance analysis */

The NLP tab contains the visone Natural Language Processing functionality which can be used to create networks from text written in a natural language (usually English).

The NLP tab is not shown by default. In order for it to be enabled, visone must be started with the command line option
-e de.visone.nlp.NLPExtension
If texts with long sentences are to be processed, it is furthermore desirable to add <code>-Xmx1g</code> to prevent visone from running out of memory during network creation.

In addition, the files <code>englishPCFG.ser.gz</code> and <code>left3words-wsj-0-18.tagger</code> should be downloaded from the [http://nlp.stanford.edu/ Stanford Natural Language Processing Group's website] and placed in the visone working directory. This will allow ''centering resonance analysis'' and ''word net analysis'' to work without explicitly specifying a parser or tagger file (see below).

== centering resonance analysis ==
This performs a [CRA] analysis of the input text.
* '''quick layout''' does a quick layout after network creation, which may take some time for huge networks.
* The '''stopwords filter''' can either use the builtin list of stopwords (common function words that can be assumed to carry no meaning for the topic of the text), or a file containing one lowercase stopword per line, with empty lines and comments (starting with <code>#</code>) ignored. Also, a minimum word length can be set, so that all words shorter than this length are always ignored.
* A comma-separated list of '''center tags''' can be specified. These are the node names as used by the parser, eg. NP (noun phrase) or VP (verb phrase). Phrases can be nested, such that an NP contains more NPs. If '''take centers from bottom level''' is selected, the inner ones are used, else the top one, thus producing a much denser network.
* The '''parser file''' must be specified, however the default path points to <code>englishPCFG.ser.gz</code> in the visone working directory, so if the file is placed there as recommended, nothing should need to be selected.

== word net analysis ==
...

== crawl link structure ==
This function captures the link structure of a website into a network. One node is created for each page, and a (directed) edge for every link from one page to another. Each node will have the following attributes:
* '''url''': the URL of the page
* '''text''': the text content of the page (if enabled)
Each edge will have an attribute named '''linktext''' containing the text of the hyperlink represented by that edge. In addition, if a link was found but not followed, it will be marked as unconfirmed.

All that is required to use the crawler is the '''url''' field, which must be filled with the URL of the page at which the crawler should start, such as <code><nowiki>http://visone.info/wiki/index.php/Main_Page</nowiki></code> for the visone wiki. After crawling has finished, the network does ''not'' get layouted to allow handling of extremely large graphs, thus all nodes will be show lying on a single point. Click the quick layout button to see the network structure.

For finer control, the following options can be adjusted:
* '''browser''': The browser that visone will masquerade as, to avoid being blocked. Select one from the list, or paste any user agent string.
* '''max depth''': The maximum length of a chain of links that will be followed. For example, 0 will not follow any links, 1 will follow all links on the start page, and 2 will follow all links on the start page and on pages referenced by the start page.
* '''page limit''': The maximum number of pages that will be ''downloaded''. This is ''not'' a limit on the number of nodes created, because nodes are created for every page that is found, even if it is never downloaded.
* '''same host only''': If selected, links to different hosts will not be followed. For example, if a page on <code>visone.info</code> links to another page on <code>visone.info</code>, this link will be followed, but a link to <code>google.com</code> will not.
* '''store text''': If selected, the text content of every page that is downloaded will be stored in the '''text''' attribute of its node.

=== crawler expert options ===
* '''crawl rule selection''': This option decides which parts of the webpage will be considered textual content.
** '''standard''' uses a default rule that should work for most websites.
** '''domain specific''' selects the rule based on the URL, so that for example only the actual text of Wikipedia articles is extracted. The name of the rule file is generated from the URL, for example, the rule file for <code><nowiki>http://www.informatik.uni-konstanz.de/algo</nowiki></code> is <code>algo.www.informatik.uni-konstanz.de.rule</code>. Some rules, such as the one for Wikipedia, are already built in.
** '''specified''' uses the rule selected in the '''crawl rule''' dropdown.
* '''cookies''': If the website requires a login, paste the HTTP <code>Cookie:</code> header value here. (Obtaining this value might be complicated, though.)
* '''ignore errors''': If this option is ''not'' selected, any error encountered during crawling, such as the common ''page not found'', will terminate the crawler. Unchecking is only recommended if there are known to be no dead links.
* '''mean delay''': Delay after a successful page download, in milliseconds.
* '''cool down after error''': Delay after a failed page download, in milliseconds.
* '''respect nofollow''': If selected, visone will observe the [http://en.wikipedia.org/wiki/Nofollow nofollow] specification, and not follow any links such marked.

== crawl text ==
The purpose of this function is to download the textual content of a singe webpage and store it in a file that can be used as an input file for the text analysis methods discussed above.

Use '''output file''' to select the name of the file to (over-)write. For the '''url''', '''browser''' and expert options, see [[#crawl link structure|above]].

NLP tab

2011-07-14T18:22:37Z

Fratz: crawler expert options and crawl text sections

The NLP tab contains the visone Natural Language Processing functionality which can be used to create networks from text written in a natural language (usually English).

The NLP tab is not shown by default. In order for it to be enabled, visone must be started with the command line option
-e de.visone.nlp.NLPExtension
If texts with long sentences are to be processed, it is furthermore desirable to add <code>-Xmx1g</code> to prevent visone from running out of memory during network creation.

In addition, the files <code>englishPCFG.ser.gz</code> and <code>left3words-wsj-0-18.tagger</code> should be downloaded from the [http://nlp.stanford.edu/ Stanford Natural Language Processing Group's website] and placed in the visone working directory. This will allow ''centering resonance analysis'' and ''word net analysis'' to work without explicitly specifying a parser or tagger file (see below).

== centering resonance analysis ==
...

== word net analysis ==
...

== crawl link structure ==
This function captures the link structure of a website into a network. One node is created for each page, and a (directed) edge for every link from one page to another. Each node will have the following attributes:
* '''url''': the URL of the page
* '''text''': the text content of the page (if enabled)
Each edge will have an attribute named '''linktext''' containing the text of the hyperlink represented by that edge. In addition, if a link was found but not followed, it will be marked as unconfirmed.

All that is required to use the crawler is the '''url''' field, which must be filled with the URL of the page at which the crawler should start, such as <code><nowiki>http://visone.info/wiki/index.php/Main_Page</nowiki></code> for the visone wiki. After crawling has finished, the network does ''not'' get layouted to allow handling of extremely large graphs, thus all nodes will be show lying on a single point. Click the quick layout button to see the network structure.

For finer control, the following options can be adjusted:
* '''browser''': The browser that visone will masquerade as, to avoid being blocked. Select one from the list, or paste any user agent string.
* '''max depth''': The maximum length of a chain of links that will be followed. For example, 0 will not follow any links, 1 will follow all links on the start page, and 2 will follow all links on the start page and on pages referenced by the start page.
* '''page limit''': The maximum number of pages that will be ''downloaded''. This is ''not'' a limit on the number of nodes created, because nodes are created for every page that is found, even if it is never downloaded.
* '''same host only''': If selected, links to different hosts will not be followed. For example, if a page on <code>visone.info</code> links to another page on <code>visone.info</code>, this link will be followed, but a link to <code>google.com</code> will not.
* '''store text''': If selected, the text content of every page that is downloaded will be stored in the '''text''' attribute of its node.

=== crawler expert options ===
* '''crawl rule selection''': This option decides which parts of the webpage will be considered textual content.
** '''standard''' uses a default rule that should work for most websites.
** '''domain specific''' selects the rule based on the URL, so that for example only the actual text of Wikipedia articles is extracted. The name of the rule file is generated from the URL, for example, the rule file for <code><nowiki>http://www.informatik.uni-konstanz.de/algo</nowiki></code> is <code>algo.www.informatik.uni-konstanz.de.rule</code>. Some rules, such as the one for Wikipedia, are already built in.
** '''specified''' uses the rule selected in the '''crawl rule''' dropdown.
* '''cookies''': If the website requires a login, paste the HTTP <code>Cookie:</code> header value here. (Obtaining this value might be complicated, though.)
* '''ignore errors''': If this option is ''not'' selected, any error encountered during crawling, such as the common ''page not found'', will terminate the crawler. Unchecking is only recommended if there are known to be no dead links.
* '''mean delay''': Delay after a successful page download, in milliseconds.
* '''cool down after error''': Delay after a failed page download, in milliseconds.
* '''respect nofollow''': If selected, visone will observe the [http://en.wikipedia.org/wiki/Nofollow nofollow] specification, and not follow any links such marked.

== crawl text ==
The purpose of this function is to download the textual content of a singe webpage and store it in a file that can be used as an input file for the text analysis methods discussed above.

Use '''output file''' to select the name of the file to (over-)write. For the '''url''', '''browser''' and expert options, see [[#crawl link structure|above]].

NLP tab

2011-07-14T17:29:05Z

Fratz: crawl link structure documentation

NLP tab

2011-07-14T16:40:49Z

Fratz: skeleton

CRA

2010-11-30T14:24:04Z

Fratz:

'''Centering Resonance Analysis''' ('''CRA''') extracts a network from a text by analysing its ''centers'',
for which the ''Centering Theory'' states that they
contain the main contents of the text.
According to Centering Theory, these centers are the ''Noun Phrases'' (''NP''s) of a text, that is the nouns together with any modifiers belonging to them. Thus, words within these centers define the words within the CRA text network, and the way they occur in the text can cause links between them.

Let us consider the following example sentence taken from the short story ''We Can Remember It for You Wholesale'' by ''Philip K. Dick'':
<blockquote>
Half an ancient silver fifty cent piece, several quotations from John
Donne's sermons written incorrectly, each on a separate piece of
transparent tissue-thin paper, ...
</blockquote>

In the first step, this sentence is parsed to extract the NPs. For each of the words appearing in an NP, a node is created in the graph.
<blockquote>
Half an ancient silver fifty cent piece, several quotations from John
Donne's sermons written incorrectly, each on a separate piece of
transparent tissue-thin paper, ...
</blockquote>

[[File:CRA0.png|700px]]

Next, words that occur within the same NP are connected, regardless of their distance within the NPs. For example, the NP ''transparent tissue-thin paper'' causes the edges between ''transparent'' and ''paper'', ''paper'' and ''tissue-thin'' as well as ''tissue-thin'' and ''transparent''. Had there been five words in the NP, each of them would have been connected with all of the four others.
<blockquote>
Half an ancient silver fifty cent piece, several quotations from John
Donne's sermons written incorrectly, each on a separate piece of
transparent tissue-thin paper, ...
</blockquote>

[[File:CRA1.png|700px]]

Words that are not part of the same NP are still connected if one of them is at the end of an NP and the other is at the beginning of the following NP. That is, because ''piece'' is the last word of then NP ''separate piece'' and ''transparent'' if the first word of the NP following it, the two words are connected in the network.
<blockquote>
Half an ancient silver fifty cent piece, several quotations from John
Donne's sermons written incorrectly, each on a separate piece of
transparent tissue-thin paper, ...
</blockquote>

[[File:CRA2.png|700px]]

Finally, duplicate nodes for the same word are merged. For example, because ''piece'' appears twice, two nodes were created for it.

[[File:CRA3.png|700px]]

These two nodes are now merged into just one.

[[File:CRA4.png|700px]]

CRA

2010-11-30T14:03:01Z

Fratz:

'''Centering Resonance Analysis''' ('''CRA''') extracts a network from a text by analysing its ''centers'',
for which the ''Centering Theory'' states that they
contain the main contents of the text.
According to Centering Theory, these centers are the ''Noun Phrases'' (''NP''s) of a text, that is the nouns together with any modifiers belonging to them. Thus, words within these centers define the words within the CRA text network, and the way they occur in the text can cause links between them.

Let us consider the following example sentence taken from the short story ''We Can Remember It for You Wholesale'' by ''Philip K. Dick'':
<blockquote>
Half an ancient silver fifty cent piece, several quotations from John
Donne's sermons written incorrectly, each on a separate piece of
transparent tissue-thin paper, ...
</blockquote>

In the first step, this sentence is parsed to extract the NPs. For each of the words appearing in an NP, a node is created in the graph.
<blockquote>
Half an ancient silver fifty cent piece, several quotations from John
Donne's sermons written incorrectly, each on a separate piece of
transparent tissue-thin paper, ...
</blockquote>

[[File:CRA0.png]]

Next, words that occur within the same NP are connected, regardless of their distance within the NPs. For example, the NP ''transparent tissue-thin paper'' causes the edges between ''transparent'' and ''paper'', ''paper'' and ''tissue-thin'' as well as ''tissue-thin'' and ''transparent''. Had there been five words in the NP, each of them would have been connected with all of the four others.
<blockquote>
Half an ancient silver fifty cent piece, several quotations from John
Donne's sermons written incorrectly, each on a separate piece of
transparent tissue-thin paper, ...
</blockquote>

[[File:CRA1.png]]

Words that are not part of the same NP are still connected if one of them is at the end of an NP and the other is at the beginning of the following NP. That is, because ''piece'' is the last word of then NP ''separate piece'' and ''transparent'' if the first word of the NP following it, the two words are connected in the network.
<blockquote>
Half an ancient silver fifty cent piece, several quotations from John
Donne's sermons written incorrectly, each on a separate piece of
transparent tissue-thin paper, ...
</blockquote>

[[File:CRA2.png]]

Finally, duplicate nodes for the same word are merged. For example, because ''piece'' appears twice, two nodes were created for it.

[[File:CRA3.png]]

These two nodes are now merged into just one.

[[File:CRA4.png]]

File:CRA4.png

2010-11-30T14:00:33Z

Fratz:

File:CRA3.png

2010-11-30T14:00:17Z

Fratz:

File:CRA2.png

2010-11-30T14:00:07Z

Fratz:

File:CRA1.png

2010-11-30T13:59:56Z

Fratz:

File:CRA0.png

2010-11-30T13:59:15Z

Fratz:

Betweenness

2010-11-29T19:50:04Z

Fratz: Created page with ''''Betweenness''' is another means to quantify the structural importance of nodes in a network. It is defined as the percentage of all shortest paths that contain a certain node…'

'''Betweenness''' is another means to quantify the structural importance of nodes in a network.

It is defined as the percentage of all shortest paths that contain a certain node.

That is, under the assumption that information, traffic etc. flows only along shortest paths, nodes are ranked according to the amount of information, traffic etc. that flows over them.

CRA

2010-11-25T22:58:44Z

Fratz:

'''Centering Resonance Analysis''' ('''CRA''') extracts a network from a text by analysing its ''centers'',
for which the ''Centering Theory'' states that they
contain the main contents of the text.
According to Centering Theory, these centers are the ''Noun Phrases'' (''NP''s) of a text, that is the nouns together with any modifiers belonging to them. Thus, words within these centers define the words within the CRA text network, and the way they occur in the text can cause links between them.

Let us consider the following example sentence taken from the short story ''We Can Remember It for You Wholesale'' by ''Philip K. Dick'':
<blockquote>
Half an ancient silver fifty cent piece, several quotations from John
Donne's sermons written incorrectly, each on a separate piece of
transparent tissue-thin paper, ...
</blockquote>

In the first step, this sentence is parsed to extract the NPs. For each of the words appearing in an NP, a node is created in the graph.
<blockquote>
Half an ancient silver fifty cent piece, several quotations from John
Donne's sermons written incorrectly, each on a separate piece of
transparent tissue-thin paper, ...
</blockquote>
http://tqzamf.ath.cx/pkd/cra0.png

Next, words that occur within the same NP are connected, regardless of their distance within the NPs. For example, the NP ''transparent tissue-thin paper'' causes the edges between ''transparent'' and ''paper'', ''paper'' and ''tissue-thin'' as well as ''tissue-thin'' and ''transparent''. Had there been five words in the NP, each of them would have been connected with all of the four others.
<blockquote>
Half an ancient silver fifty cent piece, several quotations from John
Donne's sermons written incorrectly, each on a separate piece of
transparent tissue-thin paper, ...
</blockquote>
http://tqzamf.ath.cx/pkd/cra2.png

Words that are not part of the same NP are still connected if one of them is at the end of an NP and the other is at the beginning of the following NP. That is, because ''piece'' is the last word of then NP ''separate piece'' and ''transparent'' if the first word of the NP following it, the two words are connected in the network.
<blockquote>
Half an ancient silver fifty cent piece, several quotations from John
Donne's sermons written incorrectly, each on a separate piece of
transparent tissue-thin paper, ...
</blockquote>
http://tqzamf.ath.cx/pkd/cra3.png

Finally, duplicate nodes for the same word are merged. For example, because ''piece'' appears twice, two nodes were created for it; these are now merged into just one.
http://tqzamf.ath.cx/pkd/cra4.png
http://tqzamf.ath.cx/pkd/cra5.png

CRA

2010-11-25T22:57:42Z

Fratz:

'''Centering Resonance Analysis''' ('''CRA''') extracts a network from a text by analysing its ''centers'',
for which the ''Centering Theory'' states that they
contain the main contents of the text.
According to Centering Theory, these centers are the ''Noun Phrases'' (''NP''s) of a text, that is the nouns together with any modifiers belonging to them. Thus, words within these centers define the words within the CRA text network, and the way they occur in the text can cause links between them.

Let us consider the following example sentence taken from the short story ''We Can Remember It for You Wholesale'' by ''Philip K. Dick'':
<blockquote>
Half an ancient silver fifty cent piece, several quotations from John
Donne's sermons written incorrectly, each on a separate piece of
transparent tissue-thin paper, ...
</blockquote>

In the first step, this sentence is parsed to extract the NPs. For each of the words appearing in an NP, a node is created in the graph.
<blockquote>
Half an ancient silver fifty cent piece, several quotations from John
Donne's sermons written incorrectly, each on a separate piece of
transparent tissue-thin paper, ...
</blockquote>
http://tqzamf.ath.cx/pkd/cra0.png

Next, words that occur within the same NP are connected, regardless of their distance within the NPs. For example, the NP ''transparent tissue-thin paper'' causes the edges between ''transparent'' and ''paper'', ''paper'' and ''tissue-thin'' as well as ''tissue-thin'' and ''transparent''. Had there been five words in the NP, each of them would have been connected with all of the four others.
<blockquote>
Half an ancient silver fifty cent piece, several quotations from John
Donne's sermons written incorrectly, each on a separate piece of
transparent tissue-thin paper, ...
</blockquote>
http://tqzamf.ath.cx/pkd/cra2.png

Words that are not part of the same NP are still connected if one of them is at the end of an NP and the other is at the beginning of the following NP. That is, because ''piece'' is the last word of then NP ''separate piece'' and ''transparent'' if the first word of the NP following it, the two words are connected in the network.
<blockquote>
Half an ancient silver fifty cent piece, several quotations from John
Donne's sermons written incorrectly, each on a separate piece of
transparent tissue-thin paper, ...
</blockquote>
http://tqzamf.ath.cx/pkd/cra3.png

Finally, duplicate nodes for the same word are merged.
http://tqzamf.ath.cx/pkd/cra4.png
http://tqzamf.ath.cx/pkd/cra5.png

CRA

2010-11-25T22:46:51Z

Fratz:

CRA

2010-11-16T13:26:10Z

Fratz: Created page with ''''Centering Resonance Analysis''' ('''CRA''') extracts a network from a text by analysing its ''centers'', for which the ''Centering Theory'' states that they contain the main c…'

Parse Tree

2010-11-16T12:49:03Z

Fratz:

A '''parse tree''' represents the structural construction of a sentence with respect to the grammar of the language in question.

For example, we could construct the following toy grammar for the English language using the subject-predicate-object structure.

Sentence ::= NP VP
| NP VP NP
NP ::= Noun
| DT Noun
VP ::= Verb

Here we use the conventional names of '''NP''' for ''Noun Phrases'', '''VP''' for ''Verb Phrases'' and '''DT''' for ''Determiners'' like ''the'' or ''an''.

Using this grammar, we can describe sentences like ''John sleeps'' or ''the dog eats the cake''.
For ''John sleeps'', we can use the first rule, which states that a Sentence can be an NP followed by a VP. Using the third rule, an NP can be just a noun, such as ''John'', and similarly, according to the fifth rule, a VP can consist of simply a verb, such as ''sleeps''.
Because Sentence is split into NP and VP, which are then further specialized into Noun and Verb respectively, it makes sense to draw this derivation as a tree where each grammatical entity is connected to the entity it is derived from. The parse tree for the example sentence ''John sleeps'' would thus be:

Sentence
/ \
NP VP
| |
Noun Verb
| |
''John'' ''sleeps''

As a more interesting example, let us consider the sentence ''Sammy eats a mouse'' (Sammy being our university's resident cat).
Here, the second rule is used, decomposing a Sentence into an NP (the subject), a VP (the predicate) and another NP (the object).
Clearly, both the first NP and the VP again simply derive a noun (''Sammy'') and a verb (''eats'') using the third and fifth rule respectively,
however the second NP uses the fourth rule to further split into a determiner (''the'') and a noun (''mouse'').
The parse tree thus shows the second NP decomposing into DT and Noun:

Sentence
/ | \
NP VP NP
| | / \
Noun Verb DT Noun
| | | |
''Sammy'' ''eats'' ''a'' ''mouse''

Some systems, such as the famous [http://nlp.stanford.edu/software/lex-parser.shtml Stanford Parser], instead show this tree sideways, that is, the output would look like shown below. Note that the structure is preserved, only its presentation is different and more compact; the above tree would look approximately like this:

(Sentence
(NP (Noun ''Sammy''))
(VP (Verb ''eats''))
(NP (DT ''the'')
(Noun ''mouse'')))

Using a much more sophisticated grammar, the Stanford Parser can almost perfectly describe arbitrary English text using such a tree. The Stanford-provided example sentence ''My dog also likes eating bananas'' is represented by

(ROOT
(S
(NP (PRP$ My) (NN dog))
(ADVP (RB also))
(VP (VBZ likes)
(S
(VP (VBG eating)
(S
(ADJP (NNS bananas))))))
(. .)))

Parse Tree

2010-11-16T12:27:41Z

Fratz:

A '''parse tree''' represents the structural construction of a sentence with respect to the grammar of the language in question.

For example, we could construct the following toy grammar for the English language using the subject-predicate-object structure.

Sentence ::= NP VP
| NP VP NP
NP ::= Noun
| DT Noun
VP ::= Verb

Here we use the conventional names of '''NP''' for ''Noun Phrases'', '''VP''' for ''Verb Phrases'' and '''DT''' for ''Determiners'' like ''the'' or ''an''.

Using this grammar, we can describe sentences like ''John sleeps'' or ''the dog eats the cake''.
For ''John sleeps'', we can use the first rule, which states that a Sentence can be an NP followed by a VP. Using the third rule, an NP can be just a noun, such as ''John'', and similarly, according to the fifth rule, a VP can consist of simply a verb, such as ''sleeps''.
Because Sentence is split into NP and VP, which are then further specialized into Noun and Verb respectively, it makes sense to draw this derivation as a tree where each grammatical entity is connected to the entity it is derived from. The parse tree for the example sentence ''John sleeps'' would thus be:

Sentence
/ \
NP VP
| |
Noun Verb
| |
''John'' ''sleeps''

As a more interesting example, let us consider the sentence ''Tom eats the mouse'' (with Tom being a cat).
Here, the second rule is used, decomposing a Sentence into an NP (the subject), a VP (the predicate) and another NP (the object).
Clearly, both the first NP and the VP again simply derive a noun (''Tom'') and a verb (''eats'') using the third and fifth rule respectively,
however the second NP uses the fourth rule to further split into a determiner (''the'') and a noun (''mouse'').
The parse tree thus shows the second NP decomposing into DT and Noun:

Sentence
/ | \
NP VP NP
| | / \
Noun Verb DT Noun
| | | |
''Tom'' ''eats'' ''the'' ''mouse''

Parse Tree

2010-11-16T12:24:47Z

Fratz:

A '''parse tree''' represents the structural construction of a sentence with respect to the grammar of the language in question.

For example, we could construct a toy grammar for the English language using the subject-predicate-object structure, as

Sentence ::= NP VP
| NP VP NP
NP ::= Noun
| DT Noun
VP ::= Verb

using the conventional names of '''NP''' for ''Noun Phrases'', '''VP''' for ''Verb Phrases'' and '''DT''' for ''Determiners'' like ''the'' or ''an''.

Using this grammar, we can describe sentences like ''John sleeps'' or ''the dog eats the cake''.
For ''John sleeps'', we can use the first rule, which states that a Sentence can be an NP followed by a VP. Using the third rule, an NP can be just a noun, such as ''John'', and similarly, according to the fifth rule, a VP can consist of simply a verb, such as ''sleeps''.
Because Sentence is split into NP and VP, which are then further specialized into Noun and Verb respectively, it makes sense to draw this derivation as a tree where each grammatical entity is connected to the entity it is derived from. The parse tree for the example sentence ''John sleeps'' would thus be:

Sentence
/ \
NP VP
| |
Noun Verb
| |
''John'' ''sleeps''

As a more interesting example, let us consider the sentence ''Tom eats the mouse'' (with Tom being a cat).
Here, the second rule is used, decomposing a Sentence into an NP (the subject), a VP (the predicate) and another NP (the object).
Clearly, both the first NP and the VP again simply derive a noun (''Tom'') and a verb (''eats'') using the third and fifth rule respectively,
however the second NP uses the fourth rule to further split into a determiner (''the'') and a noun (''mouse'').
The parse tree thus shows the second NP decomposing into DT and Noun:

Sentence
/ | \
NP VP NP
| | / \
Noun Verb DT Noun
| | | |
''Tom'' ''eats'' ''the'' ''mouse''

Parse Tree

2010-11-16T12:06:48Z

Fratz:

Parse Tree

2010-11-16T11:59:06Z

Fratz: Created page with 'A '''parse tree''' represents the structural construction of a sentence with respect to the grammar of the language in question. For example, we could construct a toy grammar fo…'