|Subject:||bug in lesk normalization|
Lesk normalization has always been a little unstable (and can provide scores greater than 1). The following report is from Ryan Simmons and provides more details. -------------------------------------------- The past few days I have been working with the Lesk normalization feature, which (as has been mentioned previously) doesn't always constrain the output to the upper bound of 1, as it should. I am not sure if I have found the problem, or merely another symptom of it ... I haven't had the chance to experiment that much, but I figured I would let you know what I found. I am not an expert at WordNet or Perl programming in general, so please point out my mistakes. When Lesk is calculated, function scores are obtained for various relation pairs (from the lesk-relation.dat file). The default/example file has 88 pairs (also-also, also-attr, etc.). For each pair, the overlap score is calculated (and normalized if that option is activated). The score is determined for each relation pair, then added to the main score. So, the main score is the sum of the individual scores for each relation pair within the super gloss. In the lesk.pm file, the score obtained by counting the glosses for each relation pair is normalized according to the size of glosses; however, these numbers are still added together, so the main score will exceed 1. For example: Say you compare "dog#n#1" and "dog#n#1" with Lesk. To make my example a little simpler, I used the following lesk-relation.dat file, instead of the default one: RelationFile also-also attr-attr caus-caus enta-enta example-example glosexample-glosexample glos-glos holo-holo hype-hype hypo-hypo mero-mero part-part pert-pert sim-sim syns-syns The output for this is 5.15428512949297. Now, I ran Lesk again, using 15 separate relation.dat files, each with only a single relation pair each. So, "also-also", than "attr-attr", than "caus-caus", etc. Here are the values: also-also = 0 attr-attr = 0 caus-caus = 0 enta-enta = 0 example-example = 1 glosexample-glosexample = 1 glos-glos = 1 holo-holo = 0.505190311418685 hype-hype = 0.470663265306122 hypo-hypo = 0.0584315527681661 mero-mero = 1 part-part = 0 pert-pert = 0 sim-sim = 0 syns-syns = 0.12 They, predictably, add up to 5.15428512949297. Now, this example isn't perfect (under the default relation.dat file, the score for dog#n#1 and dog#n#1 is 4.25107026707742), but I think it illustrates the issue. Since the [0,1] normalization only occurs at the relation pair level, in cases of identity it will add up to be greater than 1. I am not sure what the best workaround for this will be (or even if I have the problem really nailed down ... my example might not be a representative one, and I haven't had the time to check a whole lot more, under different/more variable conditions). But, so far as I can tell from looking at the output and the .pm file, this is where the problem is occurring.