I mean which Netfilter module returned the error message? Maybe you just needed to modprobe or insmod it first.
I am trying to evoke the multicast filter.
ebtables -A FORWARD -o "eth1" --pkttype-type multicast -j DROP
ebtables -A OUTPUT -o "eth1" --pkttype-type multicast -j DROP