Any respirator must be fit tested, Qualitatively or Quantitatively. Dust masks are those annoying buggers that don't really make a seal of any kind in the first place, even while in use. I have heard that 3m makes a kind of cup or other item that aids in the fit testing. Personally, I would invest in the half-faced respirators and the rubber seal. Disposable filters are easy to change out, and can be used for more the one kind of application. i.e, changing from a simple dust filter to an organic vapor or the like. They also are much easier to seal and fit test.
That is, of course, if the atmosphere is above the PEL of whatever your trying to be protect your workers from. Otherwise, if it's below the PEL. Fit testing doesn't really matter at all, since you don't really need it in the first place. (pretty sure that's right)
Also, are the dust masks even rated to stop the smoke from passing through? From what I understand all they do is keep dust and particles like grass clippings from being inhaled. It could be that you are getting a proper seal but the irritant is simply being inhaled through the mask.
http://www.ehso.com/dispresp.htm (Good Article)
Good luck. Again, I would go with the half faced rubber sealed masks. Lot less hassle for the time it takes to get in compliance with the dust masks.
Build a man a fire, and keep him warm for a day. Set a man on fire, and keep him warm for the rest of his life.