Bug crashes on certain inputs

[katelee168]

Ex: "eeb"

Though .Stem checks for words shorter than 3 letters before steps 1-4 execute. By the time it reaches step5a eeb becomes e and there is an index out of bounds error.

[reiver]

I cannot reproduce your error. Could you please provide some sample code to reproduce the error?

---

It created a test to try to reproduce this, but cannot.

The test I created (which passes for me) is:

func TestStemStringEeb(t *testing.T) {

    expected := "eeb"

    s := "eeb"
    actual := StemString(s)
    if expected != actual {
        t.Errorf("Input: [%s] -> Actual: [%s]. Expected: [%s]", s, actual, expected)
    }
}

[smhanov]

I also get this. I came here to let you know.

The input string that causes the crash for me is "eed"

runtime.panic(0x9415a0, 0xe94bbc)
/usr/local/go/src/pkg/runtime/panic.c:279 +0xf5
github.com/reiver/go-porterstemmer.step5a(0xc20847e090, 0x1, 0x4, 0x0, 0x0, 0x0)
/home/smhanov/canbuy/src/github.com/reiver/go-porterstemmer/porterstemmer.go:792 +0x174
github.com/reiver/go-porterstemmer.StemWithoutLowerCasing(0xc20847e090, 0x1, 0x4, 0x0, 0x0, 0x0)
/home/smhanov/canbuy/src/github.com/reiver/go-porterstemmer/porterstemmer.go:896 +0x196
github.com/reiver/go-porterstemmer.Stem(0xc20847e090, 0x3, 0x4, 0x0, 0x0, 0x0)
/home/smhanov/canbuy/src/github.com/reiver/go-porterstemmer/porterstemmer.go:868 +0xd2
github.com/reiver/go-porterstemmer.StemString(0xc2082d54e8, 0x3, 0x0, 0x0)
/home/smhanov/canbuy/src/github.com/reiver/go-porterstemmer/porterstemmer.go:839 +0x74
dexter.(_searchIndex).stem(0xc208029110, 0xc2082d54c0, 0x35, 0x0, 0x0, 0x0)
/home/smhanov/canbuy/src/dexter/search.go:177 +0x2c0
dexter.(_searchIndex).Add(0xc208029110, 0x1a497a, 0xc2082d5440, 0x35, 0x8f24e0, 0xc208153e00)
/home/smhanov/canbuy/src/dexter/search.go:273 +0xf5
main.func·040(0xc208153e00)
/home/smhanov/canbuy/src/canbuy/main.go:363 +0x6f
main.(*BuyDatabase).Listings(0xc208038110, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/home/smhanov/canbuy/src/canbuy/database.go:220 +0x1dd8
main.StartDex()
/home/smhanov/canbuy/src/canbuy/main.go:368 +0x113
created by main.main
/home/smhanov/canbuy/src/canbuy/main.go:432 +0x4ba

Crash is here:
func step5a(s []rune) []rune {

// Initialize.
    lenS := len(s)
    result := s


// Do it!
    if 'e' == s[lenS-1] {
        lenSuffix := 1

        subSlice := s[:lenS-lenSuffix]

        m := measure(subSlice)

        if 1 < m {
            result = subSlice
>>>>>>>>>>>>>>>>>>>         } else if c := subSlice[len(subSlice)-1] ; 1 == m && !( hasConsonantVowelConsonantSuffix(subSlice) && 'w' != c && 'x' != c && 'y' != c)  {
            result = subSlice
        }
    }


// Return.
    return result

}

[katelee168]

Yes, anything that is three letters and starts with ee does it for me. Unfortunately, I don't have access to the code I was using this for anymore, so I don't have the error message, but if I recall correctly, it was as smhanov stated.

[mschoch]

The test case I added shows that the following inputs result in a panic:

--- FAIL: TestStemFuzz (13.02 seconds)
    porterstemmer_fuzz_test.go:24: StemString panicked for input 'eed'
    porterstemmer_fuzz_test.go:24: StemString panicked for input 'eeds'
    porterstemmer_fuzz_test.go:24: StemString panicked for input 'eing'
    porterstemmer_fuzz_test.go:24: StemString panicked for input 'eings'

[mholt]

Ah, yeah, I would love to see that PR merged in soon 👍

Also, @smhanov -- I didn't know you were dabbling in Go! (I've mostly seen Python code from your blog.)

[mschoch]

I'm maintaining a fork now. I don't plan on doing any new development, but I do have an interest in fixing bugs. I have cleaned up a few other things, and put some continuous integration in place.

https://github.com/blevesearch/go-porterstemmer

[mholt]

@mschoch Thanks, that looks good!

[k1xme]

@mschoch very helpful! thanks!