Actor Rob Lowe accused a tax official in California of making anti-Semitic remarks during a private meeting last year over a financial dispute concerning the sale of a house.