Re: Feature Engineeriing!
On Monday 07 March 2005 16:13, Rich Cooper wrote:
> > The mistake is the same in both cases. The solution is not better
> > features, but to abandon the assumption those features organize
> > themselves in only one way.
> >
> > -Rob
>
> Your approach is very interesting Rob, and the one about
> retaining many meanings in the natural language representation
> rings a strong bell with me. I wish I knew how to take the next
> step and find a representation that preserves the many levels
> of interpretation that people place on a phrase.
Just keep examples of language usage, Rich.
And associate meaning (or grammar) with one or other way of ordering those
examples.
If ways of ordering a set of examples are always going to be inconsistent on
some level (Goedel?), the cheapest way to keep all the ways you can structure
them (all the meaning) will be to hold on to the examples and "project out"
the structure you want when you want it.
In terms of my earlier analogy, it's something like keeping the globe and
using it to project out maps when you need them.
Or keeping the population of people and lining them up according age or height
as the case demands.
We can apply this search for order in concrete ways, and ways which match our
intuitions about meaning, as my parsing demo showed. It means nothing that we
don't have labels for all the new structure we find. Labels were always a
fairly weak model for meaning. Associating meaning with order between
examples tells us more, not less.
And anyway, the new structure will not quite be anonymous, because by drawing
a parallel between meaning and grammar, we find it indexed by syntax (a given
sequence of words forces the balance of examples in a corpus to line up
according to one incompatible order or another.)
If someone finds it useful to attach permanent labels to one order or another
and standardize it for applications in some narrow domain (essentially to
"lexicalize" it), that's fine, but in the big picture, or on the fine grain
level, they are going to want to understand the processes underlying the
order they label. If only because (given the "curvature" of knowledge =
inconsistent orderings) the maps they make are only going to remain accurate
within a very narrow domain.
-Rob