Wednesday, 9 February 2011

Introducing Renderscript

[This post is by R. Jason Sams, an Android engineer who specializes in graphics, performance tuning, and software architecture. —Tim Bray]

Renderscript is a key new Honeycomb feature which we haven’t yet discussed in much detail. I will address this in two parts. This post will be a quick overview of Renderscript. A more detailed technical post with a simple example will be provided later.

Renderscript is a new API targeted at high-performance 3D rendering and compute operations. The goal of Renderscript is to bring a lower level, higher performance API to Android developers. The target audience is the set of developers looking to maximize the performance of their applications and are comfortable working closer to the metal to achieve this. It provides the developer three primary tools: A simple 3D rendering API on top of hardware acceleration, a developer friendly compute API similar to CUDA, and a familiar language in C99.

Renderscript has been used in the creation of the new visually-rich YouTube and Books apps. It is the API used in the live wallpapers shipping with the first Honeycomb tablets.

The performance gain comes from executing native code on the device. However, unlike the existing NDK, this solution is cross-platform. The development language for Renderscript is C99 with extensions, which is compiled to a device-agnostic intermediate format during the development process and placed into the application package. When the app is run, the scripts are compiled to machine code and optimized on the device. This eliminates the problem of needing to target a specific machine architecture during the development process.

Renderscript is not intended to replace the existing high-level rendering APIs or languages on the platform. The target use is for performance-critical code segments where the needs exceed the abilities of the existing APIs.

It may seem interesting that nothing above talked about running code on CPUs vs. GPUs. The reason is that this decision is made on the device at runtime. Simple scripts will be able to run on the GPU as compute workloads when capable hardware is available. More complex scripts will run on the CPU(s). The CPU also serves as a fallback to ensure that scripts are always able to run even if a suitable GPU or other accelerator is not present. This is intended to be transparent to the developer. In general, simpler scripts will be able to run in more places in the future. For now we simply leverage the CPU resources and distribute the work across as many CPUs as are present in the device.

The video above, captured through an Android tablet’s HDMI out, is an example of Renderscript compute at work. (There’s a high-def version on YouTube.) In the video we show a simple brute force physics simulation of around 900 particles. The compute script runs each frame and automatically takes advantage of both cores. Once the physics simulation is done, a second graphics script does the rendering. In the video we push one of the larger balls to show the interaction. Then we tilt the tablet and let gravity do a little work. This shows the power of the dual A9s in the new Honeycomb tablet.

Renderscript Graphics provides a new runtime for continuously rendering scenes. This runtime sits on top of HW acceleration and uses the developers’ scripts to provide custom functionality to the controlling Dalvik code. This controlling code will send commands to it at a coarse level such as “turn the page” or “move the list”. The commands the two sides speak are determined by the scripts the developer provides. In this way it’s fully customizable. Early examples of Renderscript graphics were the live wallpapers and 3d application launcher that shipped with Eclair.

With Honeycomb, we have migrated from GL ES 1.1 to 2.0 as the renderer for Renderscript. With this, we have added programmable shader support, 3D model loading, and much more efficient allocation management. The new compiler, based on LLVM, is several times more efficient than acc was during the Eclair-through-Gingerbread time frame. The most important change is that the Renderscript API and tools are now public.

The screenshot above was taken from one of our internal test apps. The application implements a simple scene-graph which demonstrates recursive script to script calling. The Androids are loaded from an A3D file created in Maya and translated from a Collada file. A3D is an on device file format for storing Renderscript objects.

Later we will follow up with more technical information and sample code.

Android 2.3.3 Platform, New NFC Capabilities

Several weeks ago we released Android 2.3, which introduced several new forms of communication for developers and users. One of those, Near Field Communications (NFC), let developers get started creating a new class of contactless, proximity-based applications for users.

NFC is an emerging technology that promises exciting new ways to use mobile devices, including ticketing, advertising, ratings, and even data exchange with other devices. We know there’s a strong interest to include these capabilities into many applications, so we’re happy to announce an update to Android 2.3 that adds new NFC capabilities for developers. Some of the features include:

  • A comprehensive NFC reader/writer API that lets apps read and write to almost any standard NFC tag in use today.
  • Advanced Intent dispatching that gives apps more control over how/when they are launched when an NFC tag comes into range.
  • Some limited support for peer-to-peer connection with other NFC devices.

We hope you’ll find these new capabilities useful and we’re looking forward to seeing the innovative apps that you will create using them.

Android 2.3.3 is a small feature release that includes a new API level, 10.
Going forward, we expect most devices shipping with an Android 2.3 platform to run Android 2.3.3 (or later). For an overview of the API changes, see the Android 2.3.3 Version Notes. The Android 2.3.3 SDK platform for development and testing is available through the Android SDK Manager.

Thursday, 3 February 2011

The Android 3.0 Fragments API

[This post is by Dianne Hackborn, a Software Engineer who sits very near the exact center of everything Android. — Tim Bray]

An important goal for Android 3.0 is to make it easier for developers to write applications that can scale across a variety of screen sizes, beyond the facilities already available in the platform:

  • Since the beginning, Android’s UI framework has been designed around the use of layout managers, allowing UIs to be described in a way that will adjust to the space available. A common example is a ListView whose height changes depending on the size of the screen, which varies a bit between QVGA, HVGA, and WVGA aspect ratios.

  • Android 1.6 introduced a new concept of screen densities, making it easy for apps to scale between different screen resolutions when the screen is about the same physical size. Developers immediately started using this facility when higher-resolution screens were introduced, first on Droid and then on other phones.

  • Android 1.6 also made screen sizes accessible to developers, classifying them into buckets: “small” for QVGA aspect ratios, “normal” for HVGA and WVGA aspect ratios, and “large” for larger screens. Developers can use the resource system to select between different layouts based on the screen size.

The combination of layout managers and resource selection based on screen size goes a long way towards helping developers build scalable UIs for the variety of Android devices we want to enable. As a result, many existing handset applications Just Work under Honeycomb on full-size tablets, without special compatibility modes, with no changes required. However, as we move up into tablet-oriented UIs with 10-inch screens, many applications also benefit from a more radical UI adjustment than resources can easily provide by themselves.

Introducing the Fragment

Android 3.0 further helps applications adjust their interfaces with a new class called Fragment. A Fragment is a self-contained component with its own UI and lifecycle; it can be-reused in different parts of an application’s user interface depending on the desired UI flow for a particular device or screen.

In some ways you can think of a Fragment as a mini-Activity, though it can’t run independently but must be hosted within an actual Activity. In fact the introduction of the Fragment API gave us the opportunity to address many of the pain points we have seen developers hit with Activities, so in Android 3.0 the utility of Fragment extends far beyond just adjusting for different screens:

  • Embedded Activities via ActivityGroup were a nice idea, but have always been difficult to deal with since Activity is designed to be an independent self-contained component instead of closely interacting with other activities. The Fragment API is a much better solution for this, and should be considered as a replacement for embedded activities.

  • Retaining data across Activity instances could be accomplished through Activity.onRetainNonConfigurationInstance(), but this is fairly klunky and non-obvious. Fragment replaces that mechanism by allowing you to retain an entire Fragment instance just by setting a flag.

  • A specialization of Fragment called DialogFragment makes it easy to show a Dialog that is managed as part of the Activity lifecycle. This replaces Activity’s “managed dialog” APIs.

  • Another specialization of Fragment called ListFragment makes it easy to show a list of data. This is similar to the existing ListActivity (with a few more features), but should reduce the common question about how to show a list with some other data.

  • The information about all fragments currently attached to an activity is saved for you by the framework in the activity’s saved instance state and restored for you when it restarts. This can greatly reduce the amount of state save and restore code you need to write yourself.

  • The framework has built-in support for managing a back-stack of Fragment objects, making it easy to provide intra-activity Back button behavior that integrates the existing activity back stack. This state is also saved and restored for you automatically.

Getting started

To whet your appetite, here is a simple but complete example of implementing multiple UI flows using fragments. We first are going to design a landscape layout, containing a list of items on the left and details of the selected item on the right. This is the layout we want to achieve:

The code for this activity is not interesting; it just calls setContentView() with the given layout:

<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
android:orientation="horizontal"
android:layout_width="match_parent"
android:layout_height="match_parent">

<fragment class="com.example.android.apis.app.TitlesFragment"
android:id="@+id/titles" android:layout_weight="1"
android:layout_width="0px"
android:layout_height="match_parent" />

<FrameLayout android:id="@+id/details" android:layout_weight="1"
android:layout_width="0px"
android:layout_height="match_parent" />

</LinearLayout>

You can see here our first new feature: the <fragment> tag allows you to automatically instantiate and install a Fragment subclass into your view hierarchy. The fragment being implemented here derives from ListFragment, displaying and managing a list of items the user can select. The implementation below takes care of displaying the details of an item either in-place or as a separate activity, depending on the UI layout. Note how changes to fragment state (the currently shown details fragment) are retained across configuration changes for you by the framework.

public static class TitlesFragment extends ListFragment {
boolean mDualPane;
int mCurCheckPosition = 0;

@Override
public void onActivityCreated(Bundle savedState) {
super.onActivityCreated(savedState);

// Populate list with our static array of titles.
setListAdapter(new ArrayAdapter<String>(getActivity(),
R.layout.simple_list_item_checkable_1,
Shakespeare.TITLES));

// Check to see if we have a frame in which to embed the details
// fragment directly in the containing UI.
View detailsFrame = getActivity().findViewById(R.id.details);
mDualPane = detailsFrame != null
&& detailsFrame.getVisibility() == View.VISIBLE;

if (savedState != null) {
// Restore last state for checked position.
mCurCheckPosition = savedState.getInt("curChoice", 0);
}

if (mDualPane) {
// In dual-pane mode, list view highlights selected item.
getListView().setChoiceMode(ListView.CHOICE_MODE_SINGLE);
// Make sure our UI is in the correct state.
showDetails(mCurCheckPosition);
}
}

@Override
public void onSaveInstanceState(Bundle outState) {
super.onSaveInstanceState(outState);
outState.putInt("curChoice", mCurCheckPosition);
}

@Override
public void onListItemClick(ListView l, View v, int pos, long id) {
showDetails(pos);
}

/**
* Helper function to show the details of a selected item, either by
* displaying a fragment in-place in the current UI, or starting a
* whole new activity in which it is displayed.
*/
void showDetails(int index) {
mCurCheckPosition = index;

if (mDualPane) {
// We can display everything in-place with fragments.
// Have the list highlight this item and show the data.
getListView().setItemChecked(index, true);

// Check what fragment is shown, replace if needed.
DetailsFragment details = (DetailsFragment)
getFragmentManager().findFragmentById(R.id.details);
if (details == null || details.getShownIndex() != index) {
// Make new fragment to show this selection.
details = DetailsFragment.newInstance(index);

// Execute a transaction, replacing any existing
// fragment with this one inside the frame.
FragmentTransaction ft
= getFragmentManager().beginTransaction();
ft.replace(R.id.details, details);
ft.setTransition(
FragmentTransaction.TRANSIT_FRAGMENT_FADE);
ft.commit();
}

} else {
// Otherwise we need to launch a new activity to display
// the dialog fragment with selected text.
Intent intent = new Intent();
intent.setClass(getActivity(), DetailsActivity.class);
intent.putExtra("index", index);
startActivity(intent);
}
}
}

For this first screen we need an implementation of DetailsFragment, which simply shows a TextView containing the text of the currently selected item.

public static class DetailsFragment extends Fragment {
/**
* Create a new instance of DetailsFragment, initialized to
* show the text at 'index'.
*/
public static DetailsFragment newInstance(int index) {
DetailsFragment f = new DetailsFragment();

// Supply index input as an argument.
Bundle args = new Bundle();
args.putInt("index", index);
f.setArguments(args);

return f;
}

public int getShownIndex() {
return getArguments().getInt("index", 0);
}

@Override
public View onCreateView(LayoutInflater inflater,
ViewGroup container, Bundle savedInstanceState) {
if (container == null) {
// Currently in a layout without a container, so no
// reason to create our view.
return null;
}

ScrollView scroller = new ScrollView(getActivity());
TextView text = new TextView(getActivity());
int padding = (int)TypedValue.applyDimension(
TypedValue.COMPLEX_UNIT_DIP,
4, getActivity().getResources().getDisplayMetrics());
text.setPadding(padding, padding, padding, padding);
scroller.addView(text);
text.setText(Shakespeare.DIALOGUE[getShownIndex()]);
return scroller;
}
}

It is now time to add another UI flow to our application. When in portrait orientation, there is not enough room to display the two fragments side-by-side, so instead we want to show only the list like this:

With the code shown so far, all we need to do here is introduce a new layout variation for portrait screens like so:

<FrameLayout xmlns:android="http://schemas.android.com/apk/res/android"
android:layout_width="match_parent"
android:layout_height="match_parent">
<fragment class="com.example.android.apis.app.TitlesFragment"
android:id="@+id/titles"
android:layout_width="match_parent"
android:layout_height="match_parent" />
</FrameLayout>

The TitlesFragment will notice that it doesn’t have a container in which to show its details, so show only its list. When you tap on an item in the list we now need to go to a separate activity in which the details are shown.

With the DetailsFragment already implemented, the implementation of the new activity is very simple because it can reuse the same DetailsFragment from above:

public static class DetailsActivity extends FragmentActivity {

@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);

if (getResources().getConfiguration().orientation
== Configuration.ORIENTATION_LANDSCAPE) {
// If the screen is now in landscape mode, we can show the
// dialog in-line so we don't need this activity.
finish();
return;
}

if (savedInstanceState == null) {
// During initial setup, plug in the details fragment.
DetailsFragment details = new DetailsFragment();
details.setArguments(getIntent().getExtras());
getSupportFragmentManager().beginTransaction().add(
android.R.id.content, details).commit();
}
}
}

Put that all together, and we have a complete working example of an application that fairly radically changes its UI flow based on the screen it is running on, and can even adjust it on demand as the screen configuration changes.

This illustrates just one way fragments can be used to adjust your UI. Depending on your application design, you may prefer other approaches. For example, you could put your entire application in one activity in which you change the fragment structure as its state changes; the fragment back stack can come in handy in this case.

More information on the Fragment and FragmentManager APIs can be found in the Android 3.0 SDK documentation. Also be sure to look at the ApiDemos app under the Resources tab, which has a variety of Fragment demos covering their use for alternative UI flow, dialogs, lists, populating menus, retaining across activity instances, the back stack, and more.

Fragmentation for all!

For developers starting work on tablet-oriented applications designed for Android 3.0, the new Fragment API is useful for many design situations that arise from the larger screen. Reasonable use of fragments should also make it easier to adjust the resulting application’s UI to new devices in the future as needed -- for phones, TVs, or wherever Android appears.

However, the immediate need for many developers today is probably to design applications that they can provide for existing phones while also presenting an improved user interface on tablets. With Fragment only being available in Android 3.0, their shorter-term utility is greatly diminished.

To address this, we plan to have the same fragment APIs (and the new LoaderManager as well) described here available as a static library for use with older versions of Android; we’re trying to go right back to 1.6. In fact, if you compare the code examples here to those in the Android 3.0 SDK, they are slightly different: this code is from an application using an early version of the static library fragment classes which is running, as you can see on the screenshots, on Android 2.3. Our goal is to make these APIs nearly identical, so you can start using them now and, at whatever point in the future you switch to Android 3.0 as your minimum version, move to the platform’s native implementation with few changes in your app.

We don’t have a firm date for when this library will be available, but it should be relatively soon. In the meantime, you can start developing with fragments on Android 3.0 to see how they work, and most of that effort should be transferable.

Wednesday, 2 February 2011

New Merchandising and Billing Features on Android Market

[This post is by Eric Chu, Android Developer Ecosystem. —Dirk Dougherty]


Following on last week’s announcement of the Android 3.0 Preview SDK, I’d like to share some more good news with you about three important new features on Android Market.

Android Market on the Web


Starting today, we have extended Android Market client from mobile devices to every desktop. Anyone can now easily find and share applications from their favorite browser. Once users select an application they want, it will automatically be downloaded to their Android-powered devices over-the-air.

Android Market on the Web dramatically expands the discoverability of applications through a rich browsing experience, suggestion-guided searching, deep linking, social sharing, and other merchandising features.

We are releasing the initial version of Android Market on the Web in English and will be extending it to other languages in the weeks ahead.

If you have applications published on Android Market, we encourage you to visit the site and review how they are presented. If you need additional information about what assets you should provide, please visit Android Market Help Center.

You can access Android Market on the Web at:

http://market.android.com/

Buyer’s Currency

Android Market lets you sell applications to users in 32 buyer countries around the world. Today we’re introducing Buyer’s Currency to give you more control over how you price your products across those countries. This feature lets you price your applications differently in each market and improves the purchase experience for buyers by showing prices in their home currencies.

We’ll be rolling out Buyer’s Currency in stages, starting with developers in the U.S. and reaching developers in other countries shortly after. We anticipate it will take approximately four months for us to complete this process.

We encourage you to watch for the appearance of new Buyer’s Currency options in the Android Market publishing console and set prices as soon as possible.

In-app Billing

After months of hard work by the Android Market team, I am extremely pleased to announce the arrival of In-app Billing on Android Market. This new service gives developers more ways to monetize their applications through new billing models including try-and-buy, virtual goods, upgrades, and more.

The In-app Billing service manages billing transactions between apps and users, providing a consistent purchasing experience with familiar forms of payment across all apps. At the same time, it gives you full control over how your digital goods are purchased and tracked. You can let Android Market manage and track the purchases for you or you can integrate with your own back-end service to verify and track purchases in the way that's best for your app.

We’ll be launching In-app Billing in stages. Beginning today, we are providing detailed documentation and a sample application to help you get familiar with the service. Over the next few weeks we’ll be rolling out updates to the Android Market client that will enable you to test against the In-app Billing service. Before the end of this quarter, the service will be live for users, to enable you to start monetizing your applications with this new capability. For complete information about the rollout, see the release information in the In-app Billing documentation.

Helping developers merchandise and monetize their products is a top priority for the Android Market team. We will continue to work hard to to make it the best marketplace for your to distribute your products. For now, we hope you’ll check out these new features to help you better deliver your products through Android Market.

Followers