Test flight woes
Come, lend your ear to my tale of woe and I’ll tell you why our iOS9 app isn’t out yet: TestFlight is broken. I’m not exaggerating. Literally it corrupted our build.
iOS9 looks like a great release. We went to WWDC earlier this year to make sure that Droplr for iOS would be ready on launch day and we’ve been working on our release ever since.
We had our beta build ready to test one month ago and put together a list of beta testers to send it out to. That was the first time we hit a wall.
What? For some reason Apple decided not to allow external testing for iOS9 apps. We suddenly went from a healthy double-digit beta tester pool to one in the low single digits. For real? This means that a healthy beta testing process is literally impossible using Apple’s tools. It’s already looking like a rough launch for app developers.
Ok, that meant that we needed to test our new release internally. More man-hours, more tracking down devices, etc. We did that.
Fast-forward to last week, the press event, the Xcode 7 GM seed was released: download, test, archive, submit. No dice. We get another error from TestFlight saying that we can’t submit software built with a beta version of iOS. I check the build version of Xcode that I’m running. It’s definitely GM. I guess it’s sit and wait.
Finally at 5:00 pm on Friday afternoon I get an email from Apple: “Submit your apps today.” The funniest part of this email was the bottom paragraph: “Read the App Review Guidelines and use TestFlight to get feedback before submitting to the App Store”. Are they really expecting me to start beta testing with TestFlight now?! 5:00pm on Friday with a 24-hour window to get my app submitted? Right…
I submit the build. It takes around 8 hours to for TestFlight to process the binary. Now it’s 1:00 AM. I submit the binary for review.
Fast-forward to the end of the weekend and I get the dreaded “New message from App Review” email in my inbox. Cue sinking feeling in the pit of my stomach.
And this is where things got weird. I’ve been through the Apple rejection runaround for years: usually it’s some violation of a new guideline that we weren’t aware of. This time our app was crashing on startup. Huh? Broken beta testing aside, I know we tested it better than that.
I download the build on TestFlight and launch it. Sure enough, it just sits at the launch screen until Springboard kills it. I check the device logs. It’s not even getting to the main() function! After more digging it looks like the binary is the wrong size! The main function isn’t even there to be called! No wonder it’s not launching. What the heck is going on?
After more frantic research and debugging I find a thread on the Apple developer forums with a crowd of exasperated developers all experiencing the same issue, all being given the runaround by Apple Support. This morning it looks we may have tracked down the source of the problem. If you have an app extension with the same name as your app, TestFlight corrupts your build. Oh. (I assume this is a bug in the new App Thinning technology, which in theory is awesome but today is the worst thing ever.)
I changed the name of our extension and uploaded a new build. Now it’s just sit and wait for TestFlight to process the build (currently around 24 hours) and hope that it doesn’t “thin” our app until it’s dead.
I’m not writing this to vent or as an excuse, I’m writing this as a plea to Apple: We spend tons of money on your devices, your conferences, building software for your stores—can you invest in giving us developer tools that work? Today TestFlight is not just broken, it’s breaking our code. It’s impossible to use as a beta testing platform and now it’s impossible to use even as a way to deliver our code to you. I’ve been writing software for your App Stores for years now and I never thought I’d pine for the good old days before iTunesConnect had TestFlight. I don’t know what’s going on behind the scenes but if you want good, bug-free products in your store, something has got to change.
And that’s about all for now.