French stemming algorithm

Links to resources

Snowball main page

The stemmer in Snowball

The ANSI C stemmer

- and its header

Sample French vocabulary

Its stemmed equivalent

Vocabulary + stemmed equivalent in two columns

Tar-gzipped file of all of the above

French stop word list

The stemmer in Snowball - MS DOS Latin I encodings

Romance language stemmers

word

stem

word

stem

continu continua continuait continuant continuation continue continué continuel continuelle continuellement continuelles continuels continuer continuera continuerait continueront continuez continuité continuons contorsions contour contournait contournant contourne contours contractait contracté contractée contracter contractés contractions contradictoirement contradictoires contraindre contraint contrainte contraintes contraire contraires contraria

=>

continu continu continu continu continu continu continu continuel continuel continuel continuel continuel continu continu continu continu continu continu continuon contors contour contourn contourn contourn contour contract contract contract contract contract contract contradictoir contradictoir contraindr contraint contraint contraint contrair contrair contrari

main mains maintenaient maintenait maintenant maintenir maintenue maintien maintint maire maires mairie mais maïs maison maisons maistre maitre maître maîtres maîtresse maîtresses majesté majestueuse majestueusement majestueux majeur majeure major majordome majordomes majorité majorités mal malacca malade malades maladie maladies maladive

=>

main main mainten mainten mainten mainten maintenu maintien maintint mair mair mair mais maï maison maison maistr maitr maîtr maîtr maîtress maîtress majest majestu majestu majestu majeur majeur major majordom majordom major major mal malacc malad malad malad malad malad

The stemming algorithm

â   à   ç   ë   é   ê   è   ï   î   ô   û   ù

a   e   i   o   u   y   â   à   ë   é   ê   è   ï   î   ô   û   ù

jouer		->		joUer
ennuie		->		ennuIe
yeux		->		Yeux
quand		->		qUand

    a i m e r     a d o r e r     v o l e r
         |...|         |.....|       |.....|

    f a m e u s e m e n t
         |......R1.......|
               |...R2....|

Search for the longest among the following suffixes, and perform the action indicated.

ance   iqUe   isme   able   iste   eux   ances   iqUes   ismes   ables   istes

delete if in R2

atrice   ateur   ation   atrices   ateurs   ations

delete if in R2

if preceded by ic, delete if in R2, else replace by iqU

logie   logies

replace with log if in R2

usion   ution   usions   utions

replace with u if in R2

ence   ences

replace with ent if in R2

ement   ements

delete if in RV

if preceded by iv, delete if in R2 (and if further preceded by at, delete if in R2), otherwise,

if preceded by eus, delete if in R2, else replace by eux if in R1, otherwise,

if preceded by abl or iqU, delete if in R2, otherwise,

if preceded by ièr or Ièr, delete if in RV

ité   ités

delete if in R2

if preceded by abil, delete if in R2, else replace by abl, otherwise,

if preceded by ic, delete if in R2, else replace by iqU, otherwise,

if preceded by iv, delete if in R2

if   ive   ifs   ives

delete if in R2

if preceded by at, delete if in R2 (and if further preceded by ic, delete if in R2, else replace by iqU)

eaux

replace with eau

aux

replace with al if in R1

euse   euses

delete if in R2, else replace by eux if in R1

issement   issements

delete if in R1 and preceded by a non-vowel

amment

replace with ant if in RV

emment

replace with ent if in RV

ment   ments

delete if preceded by a vowel in RV

Search for the longest among the following suffixes and if found, delete if preceded by a non-vowel.

îmes   ît   îtes   i   ie   ies   ir   ira   irai   iraIent   irais   irait   iras   irent   irez   iriez   irions   irons   iront   is   issaIent   issais   issait   issant   issante   issantes   issants   isse   issent   isses   issez   issiez   issions   issons   it

(Note that the non-vowel itself must also be in RV.)

Search for the longest among the following suffixes, and perform the action indicated.

ions

delete if in R2

é   ée   ées   és   èrent   er   era   erai   eraIent   erais   erait   eras   erez   eriez   erions   erons   eront   ez   iez

delete

âmes   ât   âtes   a   ai   aIent   ais   ait   ant   ante   antes   ants   as   asse   assent   asses   assiez   assions

delete

if preceded by e, delete

(Note that the ethat may be deleted in this last step must also be in RV.)

Replace final Y with i or final ç with c

If the word ends s, not preceded by a, i, o, u, è or s, delete it.

In the rest of step 4, all tests are confined to the RV region.

Search for the longest among the following suffixes, and perform the action indicated.

ion

delete if in R2 and preceded by s or t

ier   ière   Ier   Ière

replace with i

e

delete

ë

if preceded by gu, delete

(So note that ionis removed only when it is in R2 - as well as being in RV- and preceded by sor twhich must be in RV.)

If the word ends enn, onn, ett, ell or eill, delete the last letter

If the words ends é or è followed by at least one non-vowel, remove the accent from the e.

Turn any remaining I, U and Y letters in the word back into lower case.

The same algorithm in Snowball

routines (
           prelude postlude mark_regions
           RV R1 R2
           standard_suffix
           i_verb_suffix
           verb_suffix
           residual_suffix
           un_double
           un_accent
)

externals ( stem )

integers ( pV p1 p2 )

groupings ( v keep_with_s )

stringescapes {}

/* special characters (in ISO Latin I) */

stringdef a^   hex 'E2'  // a-circumflex
stringdef a`   hex 'E0'  // a-grave
stringdef c,   hex 'E7'  // c-cedilla

stringdef e"   hex 'EB'  // e-diaeresis (rare)
stringdef e'   hex 'E9'  // e-acute
stringdef e^   hex 'EA'  // e-circumflex
stringdef e`   hex 'E8'  // e-grave
stringdef i"   hex 'EF'  // i-diaeresis
stringdef i^   hex 'EE'  // i-circumflex
stringdef o^   hex 'F4'  // o-circumflex
stringdef u^   hex 'FB'  // u-circumflex
stringdef u`   hex 'F9'  // u-grave

define v 'aeiouy{a^}{a`}{e"}{e'}{e^}{e`}{i"}{i^}{o^}{u^}{u`}'

define prelude as repeat goto (

    (  v [ ('u' ] v <- 'U') or
           ('i' ] v <- 'I') or
           ('y' ] <- 'Y')
    )
    or
    (  ['y'] v <- 'Y' )
    or
    (  'q' ['u'] <- 'U' )
)

define mark_regions as (

    $pV = limit
    $p1 = limit
    $p2 = limit  // defaults

    do (
        ( v v next ) or ( next gopast v )
        setmark pV
    )
    do (
        gopast v gopast non-v setmark p1
        gopast v gopast non-v setmark p2
    )
)

define postlude as repeat (

    [substring] among(
        'I' (<- 'i')
        'U' (<- 'u')
        'Y' (<- 'y')
        ''  (next)
    )
)

backwardmode (

    define RV as $pV <= cursor
    define R1 as $p1 <= cursor
    define R2 as $p2 <= cursor

    define standard_suffix as (
        [substring] among(

            'ance' 'iqUe' 'isme' 'able' 'iste' 'eux'
            'ances' 'iqUes' 'ismes' 'ables' 'istes'
               ( R2 delete )
            'atrice' 'ateur' 'ation'
            'atrices' 'ateurs' 'ations'
               ( R2 delete
                 try ( ['ic'] (R2 delete) or <-'iqU' )
               )
            'logie'
            'logies'
               ( R2 <- 'log' )
            'usion' 'ution'
            'usions' 'utions'
               ( R2 <- 'u' )
            'ence'
            'ences'
               ( R2 <- 'ent' )
            'ement'
            'ements'
            (
                RV delete
                try (
                    [substring] among(
                        'iv'   (R2 delete ['at'] R2 delete)
                        'eus'  ((R2 delete) or (R1<-'eux'))
                        'abl' 'iqU'
                               (R2 delete)
                        'i{e`}r' 'I{e`}r'      //)
                               (RV <-'i')      //)--new 2 Sept 02
                    )
                )
            )
            'it{e'}'
            'it{e'}s'
            (
                R2 delete
                try (
                    [substring] among(
                        'abil' ((R2 delete) or <-'abl')
                        'ic'   ((R2 delete) or <-'iqU')
                        'iv'   (R2 delete)
                    )
                )
            )
            'if' 'ive'
            'ifs' 'ives'
            (
                R2 delete
                try ( ['at'] R2 delete ['ic'] (R2 delete) or <-'iqU' )
            )
            'eaux' (<- 'eau')
            'aux'  (R1 <- 'al')
            'euse'
            'euses'((R2 delete) or (R1<-'eux'))

            'issement'
            'issements'(R1 non-v delete) // verbal

            // fail(...) below forces entry to verb_suffix. -ment typically
            // follows the p.p., e.g 'confus{e'}ment'.

            'amment'   (RV fail(<- 'ant'))
            'emment'   (RV fail(<- 'ent'))
            'ment'
            'ments'    (test(v RV) fail(delete))
                       // v is e,i,u,{e'},I or U
        )
    )

    define i_verb_suffix as setlimit tomark pV for (
        [substring] among (
            '{i^}mes' '{i^}t' '{i^}tes' 'i' 'ie' 'ies' 'ir' 'ira' 'irai'
            'iraIent' 'irais' 'irait' 'iras' 'irent' 'irez' 'iriez'
            'irions' 'irons' 'iront' 'is' 'issaIent' 'issais' 'issait'
            'issant' 'issante' 'issantes' 'issants' 'isse' 'issent' 'isses'
            'issez' 'issiez' 'issions' 'issons' 'it'
                (non-v delete)
        )
    )

    define verb_suffix as setlimit tomark pV for (
        [substring] among (
            'ions'
                (R2 delete)

            '{e'}' '{e'}e' '{e'}es' '{e'}s' '{e`}rent' 'er' 'era' 'erai'
            'eraIent' 'erais' 'erait' 'eras' 'erez' 'eriez' 'erions'
            'erons' 'eront' 'ez' 'iez'

            // 'ons' //-best omitted

                (delete)

            '{a^}mes' '{a^}t' '{a^}tes' 'a' 'ai' 'aIent' 'ais' 'ait' 'ant'
            'ante' 'antes' 'ants' 'as' 'asse' 'assent' 'asses' 'assiez'
            'assions'
                (delete
                 try(['e'] delete)
                )
        )
    )

    define keep_with_s 'aiou{e`}s'

    define residual_suffix as (
        try(['s'] test non-keep_with_s delete)
        setlimit tomark pV for (
            [substring] among(
                'ion'           (R2 's' or 't' delete)
                'ier' 'i{e`}re'
                'Ier' 'I{e`}re' (<-'i')
                'e'             (delete)
                '{e"}'          ('gu' delete)
            )
        )
    )

    define un_double as (
        test among('enn' 'onn' 'ett' 'ell' 'eill') [next] delete
    )

    define un_accent as (
        atleast 1 non-v
        [ '{e'}' or '{e`}' ] <-'e'
    )
)

define stem as (

    do prelude
    do mark_regions
    backwards (

        do (
            (
                 ( standard_suffix or
                   i_verb_suffix or
                   verb_suffix
                 )
                 and
                 try( [ ('Y'   ] <- 'i' ) or
                        ('{c,}'] <- 'c' )
                 )
            ) or
            residual_suffix
        )

        // try(['ent'] RV delete) // is best omitted

        do un_double
        do un_accent
    )
    do postlude
)