UPDATE: This article was written in April 2015 and things have changed a bit (mainly for the better). Some of the advice may be out of date.

Apple’s acquisition of popular 3rd party testing service TestFlight has made things a lot easier for those developing iOS apps, who want to invite external public beta testers into their testing process. Having a full public beta, all managed through the existing Apple infrastructure which you are already using as a developer, is a great step forward.

Since November, it’s been possible to add external testers within iTunes Connect (Apple’s interface for managing app submission) with nothing more than an email address.

The premise of this is great… you can readily include up to 1,000 members of the public to test out your app, and with the tight integration with Apple’s systems, things like in-app purchases in theory ‘just work’, and actually will be free to your testing group during the beta period. You need to find the 1,000 people of course, but these can come from an existing community or mailing list for example.

It gets better… your beta testers will get an invite email directly from Apple – which lends credibility to what you’re doing – and with a bit of common sense, you can for example send a ‘heads up’ email, immediately before sending the Apple invite emails, to clue people in that ‘they will be getting an email from Apple’. You are in control of both, so you can decide the timing. This gives you an opportunity to re-market your app to your testing group – who represent your all important early ‘tribe’, and are the crowd most likely to review your app for example when it launches commercially.

We created just such an email campaign recently for the public beta launch of a Mahjong game app.

But the transfer of TestFlight to Apple has not been without issues, and there is precious little information on the all important details…

As every online entrepreneur or marketer knows, if you are sending email to potentially a thousand people, it’s very important to understand ‘nitty gritty’ details such as… When are the emails actually sent to people? Which button? How do I resend an email to people? or… My external testers aren’t getting their emails?

It’s not helped by the fact that many people simply aren’t getting these all important invite emails from Apple (more on this in a moment…) and since you can’t readily add yourself to the external testers list, and have very often used up all your ‘spare’ email addresses by this point, troubleshooting can be rather difficult.

Here’s the important details we’ve worked out which are current as of April 2015. Please comment with your own experiences and especially if you find something here out-dated. I would very much like to keep this article current  – as things will undoubtedly change (and hopefully improve…)

• The number one issue currently is that Apple’s invite emails appear to be getting widely flagged as spam. We’ve seen this in Hotmail but Gmail too. It can lead you to think there are no invite emails being sent by Apple. It’s pretty astonishing and Apple’s advice via their developer line (1-800-633-2152) is to ask users to add do_not_reply@apple.com to their approved senders. This is not exactly ideal (we ask users to check their spam folders instead), but at least Apple are aware of the problem, and in our experience are quite keen to hear of very specific reports on this. If you open a ticket via their developer line, you can help by sending them detailed information on deliverability issues.
• It’s possible to add email addresses in bulk (via CSV import), before actually inviting them (sending them the email). This allows you to check the import has worked, before clicking the all important ‘Invite’ button.
• It’s possible to import a list of CSV email addresses that do not have first and last names (the first two columns in the CSV file), but the very first row must have a first name and last name inputted, or the import will fail.
• Do use the CSV template that Apple provides – it works
• When you click Invite, after adding users, the invite email or emails are (supposed to be) sent immediately, but this does not always seem to be the case. Out of 100 emails we added via CSV import, and clicked Invite… 33 were left in an ‘Added’ status (rather than the vastly preferred ‘Invited’). Invited means what it says – an invite email has been sent. ‘Added’ simply means you’ve added the email but an invite email hasn’t been sent yet. Note you can quickly search a large External Testers list using the search function in your browser for the word ‘Added’, to see how you are doing here.
• To get around this problem you can temporarily turn off TestFlight beta testing for your build (the green radio button on the right side of the Builds screen on the Prerelease tab), wait a few seconds for the Internal/External status (on the page) to update, and then re-enable TestFlight beta testing. Crucially this only seems to send invite emails to those in an Added status, and does not resend emails to everyone else – which would be bad. When we did this with our 33 remaining people in ‘Added’ status, 7 were left in Added status afterwards. Try again and we were left with 2 in Added status, 1 and so on. It took us 4 attempts to get all 100 people into ‘Invited’ status. So it appears that around 1 in 4 people simply don’t get converted to ‘Invited’ each time you re-enable testing, at least currently.
• Turning off TestFlight beta testing and re-enabling it does not seem to have any ill-effects. Crucially, it does not seem to trip up people who got earlier invite emails, who’ve not yet installed the app. They can still install it.
• You cannot upload a CSV file that includes emails that are already in your External Testers list, it will fail gracefully, but be aware that if you are planning a quick follow up email, to your own company’s ‘heads up’ email, be careful here so you don’t get tripped up on a tight deadline.

As I come across more tips I’ll try to add them here…