Facebook Wire Protocal Buffer （protobuf）简介

The Corner
Square Engineering Blog

Introducing Wire Protocol Buffers
August23, 2013
What is Wire?
Wire is a new, open-sourceimplementation of Google's Protocol
Buffers. It'smeant for Android devices but can be used on anything that runs Java languagecode.
AtSquare, we use Protocol Buffers extensively. Protocol Buffers are a language-and platform-neutral schema for describing and transmitting data. Developerscan use the same schemas across diverse environments, including
theenvironments we care about at Square, such as Java and Ruby servers and Androidand iOS devices.
ProtocolBuffer .proto sourcefiles are human-readable and can contain comments,
so it's easy to documentyour schema right where it is defined. Protocol Buffers define a compact binarywire format that allows schemas to evolve without breaking existing clients.Typically, a code generator is used that reads .proto files,
and emits source code inthe language of your choice. This approach helps to speed development since theresulting code is expressed in the same language as the rest of yourapplication, allowing tools such as IDE autocompletion to do their job fully.
Wire Features
As webegan to run into limitations of the standard Protocol Buffer implementation inour Android apps, we made a wish list of the features we wanted for a futureimplementation:
· Messages should contain a minimumnumber of generated methods
· Messages should be clean,developer-friendly data objects:
· They should be highly readable
· They should be deeply immutable
· They should have meaningful equals, hashCode,
and toString methods
· They should support the chained
Builder pattern
· They should inherit documentationfrom the .proto sourcefiles
· Protocol Buffer enums should maponto Java enums
· Ideally, everything should be buildableusing Java-based tools
Before wedecided to build a new library we looked at several alternatives, including therecent NanoAndroid
Protocol Buffer library.While Nano Protocol Buffers generate very few methods, they didn't meet ourother goals. Ultimately, we decided to create our own library from scratch,built on Square's ProtoParser and JavaWriter libraries.
It'shandy to be able to use generated Protocol Buffer classes as full-fledged dataobjects in your app. By including equals, hashCode,
and toString methods,messages can participate in Java collections. Since the generated code is cleanand compact, stepping
into it in the debugger is not a problem. And becausecomments in your .proto files arecopied into the generated Java
source code, the documentation for your messagesand fields is right there in your IDE.
Reducing the Method Count
In thepast, Android developers attempting to use Protocol Buffers have paid a steepprice. The standard Protocol Buffer implementation (protoc)
generates at least nine methods for each optional orrequired field in your schema (variants of get, set, has,
and clear), and atleast eighteen methods for repeated fields!
Havingall this flexibility is great in non-constrained environments — whatever methodyou need is probably just a few auto-completed keystrokes away. But in Androidenvironments the Dalvik
bytecode format imposes a hard limit of 64Kmethods in a single application. For the Square Register app, generatedProtocol Buffer code was taking up a large chunk of our method space.
Byswitching to Wire, we removed almost 10,000 methods from our application andgained a lot of breathing room to add new features.
Wire Example
Considerthe classic Person protocolbuffer definition:

messagePerson{
// The customer's full name.
requiredstringname=1;
// The customer's ID number.
requiredint32id=2;
// Email address for the customer.
optionalstringemail=3;

enumPhoneType{
MOBILE=0;
HOME=1;
WORK=2;
}

messagePhoneNumber{
// The user's phone number.
requiredstringnumber=1;
// The type of phone stored here.
optionalPhoneTypetype=2[default=HOME];
}

// A list of the user's phone numbers.
repeatedPhoneNumberphone=4;
}

The Person class Wire generates is below(the complete generated code is

here):

publicfinalclassPersonextendsMessage{
/** The customer's full name. */
@ProtoField(tag=1,type=STRING,label=REQUIRED)
publicfinalStringname;

/** The customer's ID number. */
@ProtoField(tag=2,type=INT32,label=REQUIRED)
publicfinalIntegerid;

/** Email address for the customer. */
@ProtoField(tag=3,type=STRING)
publicfinalStringemail;

/** A list of the user's phonenumbers. */
@ProtoField(tag=4,label=REPEATED)
publicfinalList<PhoneNumber>phone;

privatePerson(Builderbuilder){
super(builder);
this.name=builder.name;
this.id=builder.id;
this.email=builder.email;
this.phone=immutableCopyOf(builder.phone);
}

@Overridepublicbooleanequals(Objectother){
if(!(otherinstanceofPerson))returnfalse;
Persono=(Person)other;
returnequals(name,o.name)
&&equals(id,o.id)
&&equals(email,o.email)
&&equals(phone,o.phone);
}

@OverridepublicinthashCode(){
intresult=hashCode;
if(result==0){
result=name!=null?name.hashCode():0;
result=result*37+(id!=null?id.hashCode():0);
result=result*37+(email!=null?email.hashCode():0);
result=result*37+(phone!=null?phone.hashCode():0);
hashCode=result;
}
returnresult;
}

publicstaticfinalclassBuilderextendsMessage.Builder<Person>{
// not shown
}
}

How it Works
Aninstance of a message class can only be created by a corresponding nested Builder class.
Wire generates a singlemethod per field in each builder in order to support chaining:

Personperson=newPerson.Builder()
.name("Omar")
.id(1234)
.email("omar@wire.com")
.phone(Arrays.asList(newPhoneNumber.Builder()
.number("410-555-0909")
.type(PhoneType.MOBILE)
.build()))
.build();

Wirereduces the number of generated methods by using a public final field
for each message field.Arrays are wrapped, so message instances are deeply immutable. Each field isannotated with a @ProtoField annotationproviding
metadata that Wire needs to perform serialization anddeserialization:

@ProtoField(tag=1,type=STRING,label=REQUIRED)
publicfinalStringname;

Use thesefields directly to access your data:

if(person.phone!=null){
for(PhoneNumberphone:person.phone)
if(phone.type==PhoneType.MOBILE){
sendSms(person.name,phone.number,message);
break;
}
}
}

The codeto serialize and deserialize the Person wecreated above looks like
this:

byte[]data=person.toByteArray();
Wirewire=newWire();
PersonnewPerson=wire.parseFrom(data,Person.class);

Somefeatures, such as serialization, deserialization, and the toString method
are implemented usingreflection. Wire caches reflection information about each message class thefirst time it is encountered for better performance.
Instandard Protocol Buffers, you would call person.hasEmail() to see if an
email address hasbeen set. In Wire, you simply check if person.email == null. For repeated fields such asphone, Wire
alsorequires your app to get or set a List of PhoneNumber instances
all at once, whichsaves a lot of methods.
Wiresupports additional features such as extensions and unknown fields. At present,it lacks support for some advanced features including custom options, services,and runtime introspection of schemas. These can be
added in the future as usecases for them on constrained devices emerge.
Wiredeliberately does not support the obsolete 'groups' feature.
Try it out!
Weencourage you to try Wire, contribute to the code, and let us know how it worksin your apps

Wire Protocol Buffers

“A man got to have a code!” - Omar Little
As ourteams and programs grow, the variety and volume of data also grows. Successwill turn your simple data models into complex ones! Whether your applicationis storing data to disk or transmitting it over a network,
the structure andinterpretation of that data should be clear. Consumers work best with data theyunderstand!
Schemasdescribe and document data models. If you have data, you should have a schema.

ProtocolBuffers

Google'sProtocol Buffers are built around a great schema
language:

It's cross platform and language independent. Whatever programming language you use, you'll be able to use proto schemas with your application.

Proto schemas are backwards-compatible and future-proof. You can evolve your schema as your application loses old features and gains new ones.

It's focused. Proto schemas describe your data models. That's it.

Here's asample message definition:
syntax ="proto2";

packagesquareup.dinosaurs;

optionjava_package =
"com.squareup.dinosaurs";

import"squareup/geology/period.proto";

messageDinosaur {

// Common name of this dinosaur, like"Stegosaurus".

optionalstringname =
1;

// URLs with images of this dinosaur.

repeatedstringpicture_urls =
2;

optionalsquareup.geology.Periodperiod =
5;
}
Andhere's an enum definition:
syntax ="proto2";

packagesquareup.geology;

optionjava_package =
"com.squareup.geology";

enumPeriod {

// 145.5 million years ago — 66.0million years ago.

CRETACEOUS =
1;

// 201.3 million years ago — 145.0million years ago.

JURASSIC =
2;

// 252.17 million years ago — 201.3million years ago.

TRIASSIC =
3;
}
Thisschema language is Protocol Buffers' best feature. You might even use it purelyfor documentation purposes, such as to describe a JSON API.
Protocol Buffers also defines acompact binary encoding of messages that conform to the schema. This encodingis fast to encode, fast to decode, small to transmit, and small to store. Thebinary encoding uses numeric
tags from the schema, like the 5 for period above.
Forexample, let's encode this dinosaur:
{
name:"Stegosaurus",
period: JURASSIC
}
Theencoded value is just 15 bytes:
Hex Description
0a tag: name(1), field encoding:LENGTH_DELIMITED(2). 1 << 3 | 2
0b "Stegosaurus".length()
53 'S'
74 't'
65 'e'
67 'g'
6f 'o'
73 's'
61 'a'
75 'u'
72 'r'
75 'u'
73 's'
28 tag: period(5), field encoding: VARINT(0). 5<< 3 | 0
02 JURASSIC(2)

Why Wire?

TheProtocol Buffers schema language and binary encoding are both defined byGoogle. Wire is an independent implementation from Square that's specificallydesigned for Android and Java.
For eachmessage type defined in the schema, Wire generates an immutable model class andits builder. The generated code looks like code you'd write by hand: it'sdocumented, formatted, and simple. Wire's APIs should
feel at home toprogrammers who like Effective Java.
Thatsaid, there are some interesting design decisions in Wire:

Wire messages declare public final fields instead of the usual getter methods. This cuts down on both code generated and code executed. Less code is particularly beneficial for Android programs.

Wire avoids case mapping. A field declared as picture_urls in a schema yields a Java fieldpicture_urls and not the conventional pictureUrls camel case. Though the name feels awkward at first, it's fantastic whenever you use grep or
more sophisticated search tools. No more mapping when navigating between schema, Java source code, and data. It also provides a gentle reminder to calling code that proto messages are a bit special.

Primitive types are always boxed. If a field is absent, its value is null. This is used for naturally optional fields, such as a dinosaur whose period is unknown. A field may also be null due to schema evolution: if tomorrow we add
a carnivore boolean to our message definition, today's data won’t have a value for that field.

Here's the compact generated codefor the Dinosaur message defined above:
// Code generated by Wire protocol buffer compiler, donot edit.
// Source file: squareup/dinosaurs/dinosaur.proto at 9:1
packagecom.squareup.dinosaurs;

importcom.squareup.geology.Period;
importcom.squareup.wire.Message;
importcom.squareup.wire.ProtoAdapter;
importcom.squareup.wire.WireField;
importjava.util.List;
importokio.ByteString;

publicfinalclassDinosaurextendsMessage<Dinosaur,
Dinosaur.Builder> {

publicstaticfinalProtoAdapter<Dinosaur>ADAPTER=ProtoAdapter.newMessageAdapter(Dinosaur.class);

privatestaticfinallong serialVersionUID
=0L;

publicstaticfinalStringDEFAULT_NAME="";

publicstaticfinalPeriodDEFAULT_PERIOD=Period.CRETACEOUS;

/**
* Common name ofthis dinosaur, like "Stegosaurus".
*/

@WireField(

tag=1,

adapter="com.squareup.wire.ProtoAdapter#STRING"
)

publicfinalString name;

/**
* URLs withimages of this dinosaur.
*/

@WireField(

tag=2,

adapter="com.squareup.wire.ProtoAdapter#STRING",

label=WireField.Label.REPEATED
)

publicfinalList<String> picture_urls;

@WireField(

tag=5,

adapter="com.squareup.geology.Period#ADAPTER"
)

publicfinalPeriod period;

publicDinosaur(Stringname,
List<String>picture_urls,
Periodperiod) {

this(name, picture_urls, period,
ByteString.EMPTY);
}

publicDinosaur(Stringname,
List<String>picture_urls,
Periodperiod,
ByteStringunknownFields) {

super(unknownFields);

this.name
= name;

this.picture_urls
= immutableCopyOf("picture_urls", picture_urls);

this.period
= period;
}

@Override

publicBuildernewBuilder() {

Builder builder
=newBuilder();
builder.name
= name;
builder.picture_urls
= copyOf("picture_urls", picture_urls);
builder.period
= period;
builder.addUnknownFields(unknownFields());

return builder;
}

@Override

publicbooleanequals(Objectother)
{

if (other
==this)
returntrue;

if (!(other
instanceofDinosaur))
returnfalse;

Dinosaur o
= (Dinosaur) other;

return equals(unknownFields(), o.unknownFields())

&& equals(name, o.name)

&& equals(picture_urls, o.picture_urls)

&& equals(period, o.period);
}

@Override

publicinthashCode() {

int result
=super.hashCode;

if (result
==0) {
result
= unknownFields().hashCode();
result
= result
*37+ (name
!=null? name.hashCode()
:0);
result
= result
*37+ (picture_urls
!=null? picture_urls.hashCode()
:1);
result
= result
*37+ (period
!=null? period.hashCode()
:0);

super.hashCode
= result;
}

return result;
}

publicstaticfinalclassBuilderextendscom.squareup.wire.Message.Builder<Dinosaur,
Builder> {

publicString name;

publicList<String> picture_urls;

publicPeriod period;

publicBuilder() {
picture_urls
= newMutableList();
}

/**
* Common nameof this dinosaur, like "Stegosaurus".
*/

publicBuildername(Stringname)
{

this.name
= name;

returnthis;
}

/**
* URLs withimages of this dinosaur.
*/

publicBuilderpicture_urls(List<String>picture_urls)
{
checkElementsNotNull(picture_urls);

this.picture_urls
= picture_urls;

returnthis;
}

publicBuilderperiod(Periodperiod)
{

this.period
= period;

returnthis;
}

@Override

publicDinosaurbuild() {

returnnewDinosaur(name, picture_urls, period,buildUnknownFields());
}
}
}
The Javacode to create and access proto models is compact and readable:
Dinosaur stegosaurus
=newDinosaur.Builder()
.name("Stegosaurus")
.period(Period.JURASSIC)
.build();

System.out.println("My
favoritedinosaur existed in the "+ stegosaurus.period
+"period.");
Each type has a corresponding ProtoAdapter that can encode a message tobytes
and decode bytes back into a message.
Dinosaur stegosaurus
=...
byte[] stegosaurusBytes
=Dinosaur.ADAPTER.encode(stegosaurus);

byte[] tyrannosaurusBytes
=...
Dinosaur tyrannosaurus
=Dinosaur.ADAPTER.decode(tyrannosaurusBytes);
When accessing a field, use Wire.get() to replace null values with thecorresponding
default:
Period period
=Wire.get(stegosaurus.period,
Dinosaur.DEFAULT_PERIOD);
This isequivalent to the following:
Period period = stegosaurus.period != null ?stegosaurus.period : Dinosaur.DEFAULT_PERIOD;

GeneratingCode With Wire

Wire's compiler is available viaa Maven plugin. Put .proto sources in your
project's src/main/protodirectory,then use the plugin to generate .java files.
The plugin willautomatically add the generated Java code to your project's source roots.
<build>
<plugins>
<plugin>
<groupId>com.squareup.wire</groupId>
<artifactId>wire-maven-plugin</artifactId>
<version>${project.version}</version>
<executions>
<execution>
<phase>generate-sources</phase>
<goals>
<goal>generate-sources</goal>
</goals>
<configuration>
<protoFiles>
<param>squareup/dinosaurs/dinosaur.proto</param>
<param>squareup/geology/period.proto</param>
</protoFiles>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
Wire can read .proto files from the local file systemand from within .jar files.
Thecompiler can optionally prune your schema to a subset of root types and theirtransitive dependencies. This is useful when sharing a schema between projects:a Java service and Android app may each use a subset of
a larger shared schema.
If you don't use Maven, thecompiler also has a command line interface. Just substitute wire-compiler-VERSION-jar-with-dependencies.jar with
the path to your jar.
% java -jarwire-compiler-VERSION-jar-with-dependencies.jar \
--proto_path=src/main/proto \
--java_out=out\
squareup/dinosaurs/dinosaur.proto \
squareup/geology/period.proto
Writing com.squareup.dinosaurs.Dinosaur to out
Writing com.squareup.geology.Period to out
If you use Proguard, then youneed to add keep rules. The simplest option
is totell Proguard not to touch the Wire runtime library and your generated protocolbuffers (of course these simple rules will miss opportunities to shrink andoptimize the code):
-keep class com.squareup.wire.** { *; }
-keep class com.yourcompany.yourgeneratedcode.** { *; }

Get Wire

The wire-runtime package contains runtime supportlibraries that must be included
in applications that use Wire-generated code.
WithMaven:
<dependency>
<groupId>com.squareup.wire</groupId>
<artifactId>wire-runtime</artifactId>
<version>1.8.0</version>
</dependency>
WithGradle:
compile
'com.squareup.wire:wire-runtime:1.8.0'
Snapshots of the developmentversion are available in Sonatype's snapshots repository.

Unsupported

Wire doesnot support:

Groups - they are skipped when parsing binary input data

Wire supports custom options onmessages and fields. Other custom options are ignored. Use the --no_options compiler
flag to omit optionsfrom the generated code.

FurtherDocumentation

See Google's excellentdocumentation on
the structure and syntax of proto schemas.