Activist investor Barington Capital Group plans to mount a push to change Victoria's Secret's board of directors and end a recently adopted shareholder's rights plan, according to a source.

Activist investor Barington Capital Group joins a growing chorus of dissatisfied Victoria's Secret shareholders. The firm now holds more than 1% of the lingerie retailer's stock and plans to increase its position.

Founded by James Mitarotonda, Barington previously pushed for changes at L Brands, which split into Victoria's Secret and Bath & Body Works. At Macy's, the firm wanted the department store to create a real estate unit and cut costs to boost the share price.