Hi there!

Approaching the R&D for unknown tasks will always have that element of unknown to it, however, the approach I take, that works best for me is:
1. Have a developer estimate the number of hours they think it will take to implement that piece of functionality (then you can convert these hours into a rate)
2. Add a contingency budget into the fixed price, so that it is something you call fall into should the unknown work piece go over its estimated hours (a typical contingency budget is around 10% of the overall project budget)
3. Be clear on your communications with the client, that could be as simple as saying "We've not come across a request for this functionality before, as such, we aren't too clear on the amount of work thats fully involved"
4. The more unknowns/uncertainty around the project, the higher the contingency budget

I'm Shay by the way! I have run some major projects across the software industry, working with Team Visionary, I have had the privilege to lead major successful projects for Google, Atlassian and OnePlus.

I hope this helps you, if you do require further assistance, I would love to jump into a call and provide you with support.