iMessage Stickers and Apps

Introduction

This blog is about iMessage app in iOS, We all use messaging capabilities on our iOS devices. This is a bold statement and I have no proof for it, but it’s difficult to imagine a person owning an iOS device without having sent or received messages. The main messaging application on iOS is iMessage, but it’s not the only messaging option for iOS. You can download and choose among a huge selection of various messaging applications.

Up until iOS 10, iMessage was fully closed. That is to say, it lived in its own sandbox (and still does), and did not allow any extensions to be attached to it. In iOS 10 that has changed, and web developers can finally write our own iMessage extensions that allow even more interactivity to be added to our conversations.

iMessage apps can be of two different types:

Sticker packs

This is a special, unusual kind of app that contains only images, with absolutely no code. You can create this kind of app so users can send the images to one another in iMessage. For instance, if you offer a sticker pack full of heart shapes, users can then download the app and attach those hearts to messages that they or others send. In other words, as the name implies, images can stick to messages!

Full-fledged apps

This is where you have full control over how your iMessage app works. You can do some really fun stuff in this mode, which we will review soon. For instance, you can change an existing sticker that was sent previously by one of your contacts, so that you and the person you’re chatting with can collaboratively send and receive messages to each other.

Setting Up a Sticker Pack Application

Problem

You want to create a simple iMessage application that allows your users to send stickers to each other, without writing any code.

Solution

Follow these steps:

  1. Open Xcode if it’s not already open.
  2. Create a new project. In the new project dialog, choose Sticker Pack Application and then click Next.

Enter a product name for your project and then click Next.

3. You will then be asked to save the project somewhere. Choose an appropriate location to save the project to finish this process.

4. You should now see your project opened in Xcode and then a file named Stickers.xcstickers. Click on this file and place your sticker images inside.

5. After you’ve completed these steps, test your application on the simulator and then on devices as thoroughly as possible. Once you are happy, you need to code sign and then release your app to the iMessage app store.

Discussion

With the opening up of iMessage as a platform where developers can build stand-alone apps, Apple has created a new type of store called iMessage App Store, where applications that are compatible with iMessage will show up in the list and users can purchase or download them without cost.
If you create a sticker pack app with no accompanying iOS app, your app shows up only in the iMessage App Store. If you create an iOS app with an accompanying iMessage extension (stickers), your app shows up both in the iOS App Store (for the main iOS app) and also in the iMessage App Store (for your iMessage extension).

Your stickers can be PDF, PNG, APNG (PNG with an alpha layer), JPEG, or even (animated) GIF, but Apple recommends using PNG files for the sake of quality. If you are desperate to create a sticker app but have no images to test with, simply open Finder at  /System/Library/CoreServices/CoreTypes.bundle/Contents/Resources/, then open the ICNS files in that folder with Preview.app, export those ICNS files into PNG files, and drag and drop them into your Stickers.xcstickers file in Xcode. Then build and run your project on the simulator.

Building a Full-Fledged iMessage Application

Problem

You want to build a custom iMessage application where you have full control over the presentation of your stickers and how the user interacts with them.

Solution

Create an iMessage application in Xcode by following these steps:

1. Open Xcode if it’s not already open.

2. Create a new project. In the template, window choose iMessage Application and then click Next

3. You will be asked to save your project somewhere. Do so and then you should see Xcode open up your project.

Discussion

Now that you have created your iMessage app, it’s time to learn a bit about what’s new in the Messages framework for iOS 10 SDK. This framework contains many classes, the most important of which are:

MSMessagesAppViewController

The main view controller of your extension. It gets displayed to users when they open your iMessage application.

MSStickerBrowserViewController

A view controller that gets added to the app view controller and is responsible for displaying your stickers to the user.

MSSticker

A class that encapsulates a single sticker. There is one MSStickerfor each sticker in your pack.

MSStickerView

Every sticker instance in MSSticker has to be placed inside a view to be displayed to the user in the browser view controller. MSStickerView is the class for that view.

When you build an iMessage application as we have just done, your app is then separated into two entry points:

  • The iOS app entry point with your app delegate and the whole shebang
  • The iMessage app extension entry point

This is unlike the sticker pack app that we talked about earlier in this chapter. Sticker pack apps are iMessage apps but have no iOS apps attached to them. Therefore there is no code to be written. In full-fledged iMessage apps, your app is divided into an iOS app and an iMessage app, so you have two of some files, such as the Assets.xcassets file.

Even with custom sticker pack applications, you can build the apps in two different ways:

  • Using the existing Messages classes, such as MSStickerBrowserViewController, which do the heavy lifting for you
  • Using custom collection view controllers that will be attached to your main MSMessagesAppViewControllerinstance

 

Follow these steps to program the actual logic of the app:

 

  1. Drag and drop your PNG stickers into your project’s structure, on their own and not in an asset catalog. The reason is that we need to find them using their URLs, so we need them to sit on the disk directly.
  2. Create a new Cocoa Touch class in your project that will be your MSStickerBrowserViewController
  3. Your instance of MSStickerBrowserViewControllerhas a property called stickerBrowserView of type MSStickerBrowserView, which in turn has a property named DataSource of type MSStickerBrowserViewDataSource?. Your browser view controller by default will become this data source, which means that you need to implement all the non-optional methods of this protocol, such as numberOfStickers(in:). So let’s do that now:
  4. override func numberOfStickers(in   stickerBrowserView: MSStickerBrowserView) -> Int
    
     {   return stickers.count } 
    
    override func stickerBrowserView(_ stickerBrowserView: MSStickerBrowserView, stickerAt index: Int) -> MSSticker {   return stickers[index] }

Our browser view controller is done, but how do we display it to the user? Remember our MSMessagesAppViewController? Well, the answer is through that view controller. In the viewDidLoad() function of the aforementioned view controller, load your browser view controller and add it as a child view controller:

override func viewDidLoad() {   super.viewDidLoad()      
let controller = BrowserViewController(stickerSize: .regular)      controller.willMove(toParentViewController: self)   addChildViewController(controller)     
if let vcView = controller.view {     view.addSubview(controller.view)    
vcView.frame = view.bounds     
vcView.translatesAutoresizingMaskIntoConstraints = false     vcView.leftAnchor.constraint(equalTo: view.leftAnchor).isActive = true     vcView.rightAnchor.constraint(equalTo: view.rightAnchor).isActive = true     vcView.topAnchor.constraint(equalTo: view.topAnchor).isActive = true          vcView.bottomAnchor.constraint(equalTo:    view.bottomAnchor).isActive = true   }      controller.didMove(toParentViewController: self)    }

Now press the Run button on Xcode to run your application on the simulator or device.

In this list, simply choose the Messages app and continue. Once the simulator is running, you can manually open the Messages app, go to an existing conversation that is already placed for you there by the simulator, and press the Apps button on the keyboard.

Conclusion

In this blog, I introduced you to the new Messages framework in iOS 10, which allows you to create sticker packs and applications to integrate with iMessage. We covered the basic classes you need to be aware of, including MSStickerBrowserViewController, MSMessageAppViewController, MSSticker, and  MSStickerView.

The Messages framework provides APIs to give you a large amount of control over your iMessage apps. For further reading, I would recommend checking out Apple’s Messages Framework Reference.

Text Recognition using Firebase ML Kit for Android

Firebase ML Kit Introduction

At Google I/O 2018, Google announced Firebase ML Kit, a part of the Firebase suite that intends to give our apps the ability to support intelligent features with more ease. The SDK currently comes with a collection of pre-defined capabilities that are commonly required in applications. Firebase ML Kit offers machine learning capabilities underneath a form of a wrapper, it also offers their capabilities inside of a single SDK.

 

Currently ML Kit offers the ability to:

  • Recognize text
  • Recognize landmarks
  • Face detection
  • Scan barcodes
  • Label images

Objective

Recognizing text in images, such as the text of a street sign, and recognizing the text of documents.

Recognize Text in Images with Firebase ML Kit

ML Kit has both a general-purpose API suitable for recognizing text in images, such as the text of a street sign and an API optimized for recognizing the text of documents. The general-purpose API has both on-device and cloud-based models. Document text recognition is available only as a cloud-based model.

Prerequisites

Before you proceed, make sure you have access to the following:

  • the latest version of Android Studio
  • a device or emulator running Android API level 21 or higher
  • a Google account for Firebase and Google Cloud

Create a Firebase Project

To enable Firebase services for your app, you must create a Firebase project for it. So log in to the Firebase console and, on the welcome screen, press the Add project button. In the dialog that pops up, give the project a name and press the Create project button.

 

From the overview screen of your new project, click Add Firebase to your Android app. Enter package name and other information and press the Register app button. Now downloads configuration file ( google-services.json) that contains all the necessary Firebase metadata for your app.

Configure Your Android Studio Project

  1.  Switch to the Project view in Android Studio to see your project root directory. Move the google-services.json file you just downloaded into your Android app module root directory
  2. Modify your project level build.gradle files to use Firebase.
  3. Add dependencies in app-level build.gradle:
  4. Finally, press “Sync now”.
  5. Add permissions in AndroidManifest.xml

On Device Text Recognition

To recognize text in an image, create a FirebaseVisionImage object from either a Bitmap, media.Image, ByteBuffer, byte array, or a file on the device. Then, pass the FirebaseVisionImage object to the FirebaseVisionTextRecognizer’s processImage method. If the text recognition operation succeeds, a FirebaseVisionText object will be passed to the success listener. A FirebaseVisionText object contains the full text recognized in the image and zero or more TextBlock objects. Each TextBlock represents a rectangular block of text, which contains zero or more Line objects. Each Line object contains zero or more Element objects, which represent words and word-like entities (dates, numbers, and so on).

Demo: https://drive.google.com/openid=1XEgRMyGbQEprvM3F8h9jGrLOPyRvAZpB

On Cloud Text Recognition

If you want to use the Cloud-based model, and you have not already enabled the Cloud-based APIs for your project, do so now. Navigate to ML Kit section of the Firebase console. If you have not already upgraded your project to a Blaze plan, click Upgrade to do so. Only Blaze-level projects can use Cloud-based APIs. If Cloud-based APIs aren’t already enabled, click Enable Cloud-based APIs.

 

The document text recognition API provides an interface that is intended to be more convenient for working with images of documents on the cloud. To recognize text in an image, create a FirebaseVisionImage object from either a Bitmap, media.Image, ByteBuffer, byte array, or a file on the device. Then, pass FirebaseVisionImage object to FirebaseVisionDocumentTextRecognizer’s processImage method. If the text recognition operation succeeds, it will return a FirebaseVisionDocumentText object. A FirebaseVisionDocumentText object contains the full text recognized in the image and a hierarchy of objects (blocks, paragraph, word, symbol) that reflect the structure of the recognized document.

Demo: https://drive.google.com/openid=1qIfLKV3Mz4MJDHUf5A17uT_0c1KWz_hK

 

Stay tuned for my next article.

Build your own custom Android ROM using Android Open Source Project(AOSP)

Introduction

One of the best things about Android is custom ROMs. A custom Android ROM refers to a phone’s firmware, based on Google’s Android platform. The term ROM, which stands for Read Only Memory, really has very little to do with what a custom Android ROM actually is, can be confusing. Since Android is an open source mobile operating system that means anyone can download the source code, make modification to it, recompile it and release it for a wide variety of devices. Anyone can install ROMs to their device and achieve a modified appearance and behavior. Continue reading Build your own custom Android ROM using Android Open Source Project(AOSP)

Firebase cloud messaging in iOS

Cloud messaging or push notification is one of those “topics” that gets left out. Primarily because we are too busy beautifying the app, or working on a new feature, or we think it isn’t a big deal. Push notifications are as big a deal as any. Whether you want to re-engage your users, or deliver personalised content, or display targeted advertisements, push notification is the way to go. Continue reading Firebase cloud messaging in iOS

Go ServerLess with Firebase cloud functions

Firebase Cloud function

With announcement of cloud functions beta at Google cloud next 2017 event, Google has added one of the highly requested features in the firebase suite. This is one major step from Google in making firebase serverless. In this post, we will see some of the capabilities, pros and cons, setup and deployment of firebase cloud functions. Google IO is just days away and knowing about firebase is surely going to help in understanding the upcoming firebase features. Continue reading Go ServerLess with Firebase cloud functions

Interface Design for MoSync applications

Introduction

“Interface Design for MoSync applications” … doesn’t sound very interesting! As a matter of fact, it isn’t. There is not much new to what I am going to discuss here. The same age old problem, the same old solution, what’s new is implementing the solution for MoSync. This blog post is based more on personal experience of working with MoSync. What’s MoSync? I’ll talk about MoSync more in the next section, for now you can think of a powerful-yet-not-much-popular cross platform mobile application development framework.

Continue reading Interface Design for MoSync applications

MobileJira launched at the Windows App Store

Talentica’s Mobile Team recently launched the Windows version of the popular MobileJira App. The App, which is already available in the Android Play Store, is useful for all IT professionals who use Jira to track issues, bugs, tasks or deadlines for their projects.

Loaded with unique features, MobileJira is designed for simplicity and ease of use. It is targeted towards improving productivity and process compliance of testers, developers and managers. It helps manage JIRA tickets on the go for requirements like progress-update/resolve or close issues. It also supports major functionalities like search, issue details, updating work log/comments and view attachment.

MobileJira supports JIRA® 5.0 or later. It’s based on JIRA® REST API’s and requires the JIRA® server to run as a service. More details are available at Running JIRA as a Service.

Visit the Windows App Store to download the app.

Feel free to send in your feedback or suggestions at support-mobilejira@talentica.com to help us further improve the app.

Cross Platform Tools – Choosing Right One for Your Mobile App

Different cross platform technologies have emerged to overcome the challenge of building native build for different operating systems. Although cross platform technologies come as good alternative, every platform has its own set of limitations. The big question is to identify an appropriate tool based on requirements. In this blog post we would address this problem by comparing some of the popular cross platform toolsWe will look at Titanimum, Kony and MoSync as they are some of the most popular cross platform tools being used today. We will be doing a comparative analysis of these tools based on the ease of development, memory footprint analysis and App performance.

Ease of Development

Titanium has powerful widgets and a rich platform. It provides eclipse based studio and uses JS for development. Titanium provides good documentation and a strong developer community. It also supports device debugging. It uses native SDK for testing the app on emulator. Disadvantages include its lack of support for Windows 8.
As an alternative tool MoSync uses both C++ and HTML 5/JS as the development language. Mosync Reload can be used to test changes on multiple devices and simulators. MoSync also provides extensive documentation. MoSync doesn’t provide debugging option for building native UI. It requires native SDK for testing native UI apps on emulator.
Kony, as compared to the above two, provides a strong IDE for developing apps. With the advanced drag and drop feature like native platforms, Kony stands out from other cross platform tools. It supports third party widget libraries like Sencha and JQuery Mobile. Kony provides strong support for Enterprise based apps. In spite of all advantages, Kony’s adoption remains low because it is a paid tool. The developer community is not strong and it takes a lot of time to build apps for platforms like BB.

Memory Footprint Analysis

Mobile users are sensitive about sizes of application. Different cross platform frameworks uses plugins to access native features which in turns increases size of executables.
Platform
Executable Size
Android Native
~ 1 MB
iOS Native
~ 1 MB
Titanium
~ 8 MB
MoSync
~ 2 MB
Kony
~ 6 MB
Both Kony and Titanium use a plugin to access native features in their build but MoSync generates and uses native widgets. For a very large size application these figures may be irrelevant but for simple Productivity category app size of the executable can become an issue.

Performance

We have seen a big improvement of performance of Apps running on cross platform tools in recent years. Earlier doing some native actions or showing animation affected the app performance, mainly on Android. Even today, performance of a native build is better than any app built on cross platform tools.

We tested performance for a large List View and Switching/Loading between pages and observed that for Titanium , Mosync and Kony. Performance is good and it is on par with native builds.

In a nutshell

The choice of platform depends on your requirements. MoSync can be preferred choice if existing dev expertise is in C/C++ and also for cases where application needs to be deployed for larger set of Mobile Platforms. For enterprise based apps, Kony can be a better. If your target platform is only Android and iOS then Titanium should be preferred. None of these platforms support Game development and they better be avoided if your App relies extensively on native features.

All cross platform tools still have to go long-way to become par with native platforms like Android and iOS in term of ease of development and performance.

MobileJira Launched for the Android Market!

Talentica’s Mobile Team launched an extremely useful app for all IT professionals who use Jira to track issues, bugs, tasks or deadlines for their projects.

With its unique features, MobileJira is designed for simplicity and ease of use. It is targeted towards improving productivity and process compliance of Testers, Developers and Managers. It is the only offering that provides features such as creating issues, filtering issue list, supporting multiple users, adding/deleting logs and finally resolving or closing issues in its free version.  In fact unlike any other paid or free app, MobileJira supports

•    Custom fields
•    Adding attachments from Gallery, in-app files picker or camera
•    Saving templates for fast issue creation

MobileJira supports JIRA® 4.2.1 or later. Complete features are supported with JIRA® 5.0 on-wards. MobileJira is based on JIRA® REST API’s and requires your JIRA® server to run as service. More details are available at https://confluence.atlassian.com/display/JIRA/Running+JIRA+as+a+Service

Visit the Google Play Store to download the app.

Feel free to send in your feedback or suggestions at support-mobilejira@talentica.com to help us improve further.