Descriptors FAQ

Saturday, November 25, 2006

36. How do I use RBuf to read from a file?

This was posted by Simo as a comment here:

RBuf is also the best choice when reading strings from file. Using RBuf instead of HBufC requires less code. Also, error handling becomes more simpler.

Here is simple example how to read from RReadStream:

in .h: RBuf iSomeSetting;
in .cpp: iSomeSetting.CreateL( stream, KMaxTInt );

the HBufC would be:
in .h: HBufC* iSomeSetting;
in .cpp: iSomeSetting = HBufC::NewL( stream, KMaxTInt );
TPtr tmp( iSomeSetting->Des() );
stream >> tmp;

Error handling is also easier because there are no NULL pointer checks required for RBuf.

# posted by Jo Stichbury @ 1:26 PM 0 comments

Sunday, May 21, 2006

References to other descriptors resources

# posted by Jo Stichbury @ 2:16 PM 0 comments

Wednesday, April 26, 2006

10,000 hits!

My hit counter (www.statcounter.com) informs me that this blog clocked over 10,000 hits today. Thanks everyone for visiting over the last few months!

The blogs have been running for about a year now and are overdue for a makeover. I'm hoping to make some changes to the format and include more regular updates and new content in the near future. This will probably include moving them to a new home. But don't worry, the current content won't be removed and there will be plenty of notice of any changes.

Besides this blog, I'd just like to flag another descriptors resource on the web. Take a look at this presentation on the Symbian site, for another view of descriptors.

Best wishes,

Jo
Vancouver, April 26th 2006

# posted by Jo Stichbury @ 12:52 PM 1 comments

Saturday, August 06, 2005

35. How do I use TLex?

Well firstly, what is TLex?

It's a lexical analysis class and the lexer example code, which comes with the Symbian OS SDK, gives one example of how to use it. You can find a brief description of the lexer example in the Symbian Developer Library documentation which comes with each SDK; it is also available on the Symbian website (see the References section for details).

And, in case you're in some doubt as to what the lexer example is doing - I did - [reference 8] is a very useful resource about Reverse Polish Notation.

The rest of this article will summarise the main features of TLex into groups of useful API calls. I'd like to create another, simple, example of how to use the class, but am a bit stuck for ideas of what would be useful. If anyone's got any thoughts on how best to illustrate the main features, please do add a comment to this post and I'll see what I can do.

The main class, TLex, is like the descriptors in that two classes are available, TLex8 and TLex16 and the neutral version, TLex, is a typedef to TLex16 for today's UNICODE builds of Symbian OS. You can find the Symbian API reference documentation for TLex16 in [reference 9].

As the documentation states, TLex "...provides general string-parsing functions suitable for numeric format conversions and syntactical-element parsing...An instance of this class stores a string, maintaining an extraction mark to indicate the current lexical element being analysed and a pointer to the next character to be examined..."

TLex can be constructed with the data for lexical analysis or constructed empty and later assigned the data. Both construction and assignment take either another TLex object, a 16 bit non-modifiable descriptor or a TUint16* pointer to string data.

At the very simplest level, when the string contains just numerical data, the descriptor contents can be converted to an integer by using the Val() function of TLex. For example:

_LIT(KTestString1, "54321");

TLex lex(KTestString1());
TInt value = 0;
User::LeaveIfError(lex.Val(value));
ASSERT(value==54321);

The Val() function is overloaded for different signed integer types: TInt, TInt8, TInt16, TInt32, TInt64 - with or without limit checking. Likewise, there are Val() overloads for the unsigned integer types, passing in a radix type (decimal, hexadecimal, binary or octal). There are also overloads of Val() for TReal.

However, there's a lot more you can do with TLex. The Symbian Developer Library has more information about lexical analysis with TLex - see [reference 10] for more information.

For example, you can move through the string using the Inc() functions, or just inspect each character using Peek(). Calling Get() will both increment the position and return the character - it can be reversed using UnGet().

You can skip whitespace using SkipSpace() or characters using SkipCharacters(). The end of the string is hit when EoS() returns ETrue.

You can put an extraction marker at position in the string, using the Mark() overloads. Marking is also useful if you want to reverse (UnGetToMark() or skip using SkipAndMark() or SkipSpaceAndMark()).

Tokens are used to describe a character string which is deliminated by white space. There are a number of token methods available, such as TokenLength(), MarkedToken() - which extracts a token - and NextToken(). If you don't use tokens, you can use offsets and remainders instead - there are a number of methods for navigating the string in this way.

# posted by Jo Stichbury @ 1:18 PM 4 comments

Monday, June 27, 2005

34. Is there a way to get binary data in a _LIT?

Check out the answer to FAQ-0814 on Symbian's Developer FAQ & Tech Tips site:

Is there a way to get binary data in a _LIT?

# posted by Jo Stichbury @ 5:20 PM 0 comments

Sunday, June 26, 2005

33. Can you give an example of how to use RBuf?

Yes. I was initially reluctant to do so, because there isn't any published example code out there yet. But I'll take the plunge with this example, which shows how to use RBuf16. I'll leave code to use RBuf8, and to use the non-leaving Create() methods as an exercise for the reader, for now.

void ExampleRBufCodeL()
{
__UHEAP_MARK;
// RBuf::CreateL
RBuf modifiableBuf;
modifiableBuf.CreateL(12);
ASSERT(modifiableBuf.Length()==0);
ASSERT(modifiableBuf.MaxLength()==12);
... // Do stuff. First push modifiableBuf onto the cleanup stack if a leave may occur
modifiableBuf.Close();

// RBuf::CreateMaxL
modifiableBuf.CreateMaxL(12);
ASSERT(modifiableBuf.Length()==12);
ASSERT(modifiableBuf.MaxLength()==12);
... // Do stuff. First push modifiableBuf onto the cleanup stack if a leave may occur
modifiableBuf.Close();

// RBuf::CreateL passing in a descriptor
_LIT(KHelloWorld, "Hello World");
modifiableBuf.CreateL(KHelloWorld());
ASSERT(modifiableBuf.Length()==11);
ASSERT(modifiableBuf.MaxLength()==11);
... // Do stuff. First push modifiableBuf onto the cleanup stack if a leave may occur
modifiableBuf.Close();

// RBuf::CreateL passing in a descriptor and a maximum length
modifiableBuf.CreateL(KHelloWorld(), 15);
ASSERT(modifiableBuf.Length()==11);
ASSERT(modifiableBuf.MaxLength()==15);
... // Do stuff. First push modifiableBuf onto the cleanup stack if a leave may occur
modifiableBuf.Close()

// RBuf::CreateL and ReAllocL & modifiable descriptor base class methods
_LIT(KHello, "Hello");
_LIT(KWorld, " World");
modifiableBuf.CreateL(5);
modifiableBuf.Copy(KHello());
modifiableBuf.CleanupClosePushL(); // Push onto cleanup stack for leave safety
modifiableBuf.ReAllocL(11);
modifiableBuf.Append(KWorld);
CleanupStack::PopAndDestroy(); // Calls modifiableBuf.Close()

// RBuf::Assign
HBufC* hBuf = KHello().AllocL();
modifiableBuf.Assign(hBuf);
ASSERT(modifiableBuf.Length()==5);
... // Do stuff. First push modifiableBuf onto the cleanup stack if a leave may occur
modifiableBuf.Close();

// RBuf::Assign, ReAllocL and use of TDes::Append
TUint16* ptr = static_cast(User::AllocL(5*sizeof(TText)));
modifiableBuf.Assign(ptr,5);
ASSERT(modifiableBuf.Length()==0);
modifiableBuf.Copy(KHello()); // Copying any more would panic
modifiableBuf.CleanupClosePushL(); // Push onto cleanup stack for leave safety
modifiableBuf.ReAllocL(12);
modifiableBuf.Append(KWorld);
CleanupStack::PopAndDestroy(); // Calls modifiableBuf.Close()

__UHEAP_MARKEND;
}

# posted by Jo Stichbury @ 7:28 PM 12 comments

32. How do I use RBuf? What is it?

From the Symbian OS 8.1a SDK

RBuf16

16-bit resizable buffer descriptor [Jo comments - an equivalent 8-bit version, RBuf8, also exists]

The class provides a buffer that contains, accesses and manipulates TUint16 data. The buffer itself is on the heap, and is managed by the class.

Internally, RBuf16 behaves in one of two ways:

* as a TPtr16 descriptor type, where the buffer just contains data,
* as a pointer to a heap descriptor, an HBufC16* type, where the buffer contains both descriptor information and the data.

Note that the handling of the distinction is hidden from view.

An RBuf16 object can allocate its own buffer. Alternatively, it can take ownership of a pre-existing section of allocated memory, or it can take ownership of a pre-existing heap descriptor. It can also reallocate the buffer to resize it. Regardless of the way in which the buffer has been allocated, the RBuf16 object is responsible for freeing memory when the object itself is closed.

The class is intended for instantiation.
The class is derived from TDes16, which means that data can be both accessed and modified. The base classes provide the functions through which the data is accessed. In addition, an RBuf16 object can be passed to any function that is prototyped to take a TDes16 or a TDesC16 type.

So what does this mean for me?

An RBuf object is rather like an HBufC in that it can be created dynamically by specifying the maximum length required. However, it also benefits from being modifiable, since it derives from TDes16. This means that you don’t have to create a TPtr around the data in order to modify it, which makes it preferable to HBufC when you know you need to dynamically allocate a descriptor, and later modify it.

You don’t need to worry too much about how it’s represented internally – either as a TPtr or HBufC. In fact, all you need to know is how to create and destroy an RBuf, because calling the other descriptor operations on it should be second nature, given that you have access to all the base class methods of TDes16 and TDes16C.

But you must remember that, although RBuf manages the descriptor buffer by freeing it when you call Close(), it doesn’t manage the size of the buffer and reallocate it when you need more memory for any particular operation.

So, for example, if you call Append() on a RBuf object for which there is insufficient memory available, a panic will occur – the RBuf object will not automatically reallocate the buffer. This should be clear from the fact that the base class methods are non-leaving, that is, there is no scope for the reallocation to fail in the event of low memory.

So you still need to manage the memory for descriptor operations that may need to extend the descriptor.

The RBuf class was first introduced in Symbian OS v8.0, but first documented in the 8.1 SDK and is used extensively in software written for devices based on v9.x. The RBuf class is ideal for handling HBufC using a stack-based R class object, and may often be preferable to the contortions required to modify an HBufC.

I've created a short example of how to use the class in 33. Can you give an example of how to use RBuf?

I'd like to acknowledge JP's help and advice with this post.

# posted by Jo Stichbury @ 6:54 PM 0 comments

Friday, June 24, 2005

31. How do I convert between Java strings and descriptors?

Check out the answer to FAQ-0277 on Symbian's Developer FAQ & Tech Tips site:

"How do I convert between Java strings and Unicode descriptors?"

# posted by Jo Stichbury @ 1:42 PM 0 comments

30. How do I convert an 8-bit descriptor to a Java string?

Check out the answer to FAQ-0298 on Symbian's Developer FAQ & Tech Tips site:

"How do I convert an 8-bit descriptor to a Java string?"

These days, it's not necessary to use the Symbian cast macros (eg REINTERPRET_CAST) since these are simply defined as the equivalent C++ standard (eg reinterpret_cast). They're a legacy from when GCC didn't support the casts.

# posted by Jo Stichbury @ 11:52 AM 0 comments

Friday, June 17, 2005

29. How can I write the most efficient code when using HBufC?

(1) Spawn HBufC from existing descriptors using TDesC::AllocL() rather than creating and copying into them

HBufC can be spawned from a existing descriptor using the Alloc() or AllocL() overloads of by TDesC. You can thus replace code like this:

void SomeFunctionL(const TDesC& aDes)
{
HBufC* heapBuffer = HBufC::NewL(aDes.Length());
TPtr ptr(heapBuffer->Des());
ptr.Copy(aDes);
...
}

With the more efficient single line:

void SomeFunctionL(const TDesC& aDes)
{
HBufC* heapBuffer = aDes.AllocL();
...
}

(2) Don't call HBufC::Des() if you need only non-modifiable access to the data

It is clearer and more efficient simply to dereference an HBufC if you need a non modifiable reference to its data. You only need to call Des() when you need to modify it - Des() will return a TPtr. That is:

void Log(const TDesC& aLogBuf); // Forward declaration
void Convert(TDes& aBuf);

void CMyClass::SomeFunction()
{// iHeapBuf is a member of CMyClass
// Get non-modifiable access (TDesC&) to iHeapBuf
Log(*iHeapBuf); // Call a method which takes const TDesC&

// Now get modifiable access (TDes&) to iHeapBuf
Convert(iHeapBuf->Des()); // Call a method which takes TDes&
}

By the way, an interesting side-effect of creating a modifiable descriptor from a non-modifiable heap descriptor is discussed in Tip 9.

# posted by Jo Stichbury @ 2:29 PM 0 comments

28. How do I know what the equivalent C string library functions are for descriptors?

Just about all the C string library functionality has been implemented by the descriptor base classes, so you can chop them up, rearrange, tokenise and format them just like you have always been able to do with C strings.

Here's a useful table which compares C string functions with their descriptor equivalents. I found the information in section 5 of [reference 7]

# posted by Jo Stichbury @ 11:38 AM 1 comments

Monday, June 13, 2005

27. How do I know which kind of descriptor to use?

This flowchart may help. It's taken from my book, Symbian OS Explained (page 76), with some modification:

# posted by Jo Stichbury @ 4:15 PM 5 comments

Sunday, June 12, 2005

26. How do I convert between 8 bit and 16 bit descriptors?

TDes defines a set of Copy() methods to copy data from another descriptor of either width, from a pointer to a block of data of a specified length or from a NULL-terminated string. There are also methods for copying with folding,collation or case adjustment.

In each case, the data is copied *from* the source specified in the parameter *into* the modifiable, target, descriptor upon which the method is called. If the length of the 'incoming' data is greater than the target descriptor, a USER 11 panic occurs.

// From the definition of TDes16 in e32des16.h

IMPORT_C void Copy(const TDesC8& aDes); // Copies an 8 bit descriptor
IMPORT_C void Copy(const TDesC16& aDes); // Copies a 16 bit descriptor
IMPORT_C void Copy(const TUint16* aBuf, TInt aLength); // Copies aLength characters from the aBuf pointer
IMPORT_C void Copy(const TUint16* aString); // Copies the NULL terminated aString
IMPORT_C void CopyF(const TDesC16& aDes); // Copies and folds
IMPORT_C void CopyC(const TDesC16& aDes); // Copies and collates
IMPORT_C void CopyLC(const TDesC16& aDes); // Copies and converts the text to lower case
IMPORT_C void CopyUC(const TDesC16& aDes); // Copies and converts the text to upper case
IMPORT_C void CopyCP(const TDesC16& aDes); // Copies and capitalizes the text

Note that, because the Copy() method is overloaded to take either an 8- or 16-bit descriptor, it is possible to copy:

a narrow descriptor onto a narrow descriptor (TDes8 -> TDes8),
a wide descriptor onto a wide descriptor (TDes16 -> TDes16),
a narrow descriptor onto a wide descriptor (TDes8 -> TDes16),
a wide descriptor onto a narrow descriptor (TDes16 -> TDes8).

Copying between descriptors of equivalent character widths is fairly self-explanatory, but copying between descriptors of different character widths is worthy of elaboration:

The Copy() method implemented by TDes16 to copy an 8 bit descriptor parameter will pad each incoming character with a trailing zero.

// Instantiate a wide descriptor
_LIT16(KFred, "Fred");
TBuf16<4> wideFred(KFred); // Bytewise = F\0R\0E\0D\0

// Instantiate a narrow descriptor
_LIT8(KBert, "Bert");
TBuf8<5> narrowBert(KBert); // Bytewise = BERT

// Copy the contents of the narrow descriptor into the wide descriptor
wideFred.Copy(narrowBert); // Bytewise = B\0E\0R\0T\0

The Copy() method implemented by TDes8 to copy an incoming 16-bit descriptor will strip out alternate characters which are assumed to be zeroes. This means that this form of copy will only work if the characters in the wide string do not exceed 255 (decimal).

// Instantiate a wide descriptor
_LIT16(KFred, "Fred");
TBuf16<4> wideFred(KFred); // Bytewise = F\0R\0E\0D\0

// Instantiate a narrow descriptor
_LIT8(KBert, "Bert");
TBuf8<5> narrowBert(KBert); // Bytewise = BERT

// Copy the contents of the wide descriptor into the narrow descriptor
narrowBert.Copy(wideFred); // Bytewise = FRED

Copy() thus forms a simple way to copy and convert between narrow and wide descriptors when the character set is encoded by one 8-bit byte per character and the last byte of each wide character is simply a NULL character padding.

If you need to do a proper conversion between 16-bit Unicode and 8-bit, non-Unicode, character sets (or between Unicode and the UTF-7 and UTF-8 transformation sets), use the Symbian OS conversion library, charconv.lib.

# posted by Jo Stichbury @ 6:49 PM 2 comments

25. What do Length(), MaxLength(), SetLength() and SetMax() do?

TDesC::Length() returns the number of characters the descriptor contains.

[While we're on the subject of descriptor length, beware! Be careful not to confuse Size() and Length(). They do similar things, but while Length() returns the number of characters the descriptor contains, Size() returns the number of bytes it occupies. For 8-bit descriptors, where each character occupies a byte, this is the same thing. However, on all releases of Symbian OS since v5u, the native character width has been 16 bits, so each character occupies two bytes. Unless you're working with a very old SDK, you'll find that Size() always returns a value which is double that of Length()].

TDes::MaxLength() returns the maximum length allowed for a modifiable descriptor.

TDes::SetLength() can be used to change the descriptor's length to any value between zero and its maximum length. You can also use TDes::Zero() to set the length to zero. And TDes::SetMax() to set the length to the maximum.

[SetMax() has a slightly misleading name. It doesn’t allow you to change the maximum length of the descriptor, which would, in effect, allow you to resize its data area].

# posted by Jo Stichbury @ 5:59 PM 0 comments

Saturday, May 28, 2005

23. Why is the _L literal descriptor macro deprecated?

Why is the original _L macro now deprecated in all but test code? These have the advantage of being used in place rather than declared separately, for example:

User::Panic(_L("TEST SERVER"), KErrNotOVerflow);

You will still see them used in test code since they saves typing and thinking up a name for each literal. It's OK to do so where memory use is not critical, but you should avoid using them in production code. Why?

Well, the text, for example "TEST SERVER" is built into ROM as a basic, NULL-terminated C style string, with no initial length member. Because the layout of the stored literal is not like that of a descriptor, it means that when each instance of the literal is used, a temporary TPtrC must be constructed around the data in ROM. The construction of the temporary, which requires 4 bytes of memory for the length/type data, and instructions to set the pointer, the length and the descriptor type, is an unnecessary overhead when compared to _LIT descriptors which can be used directly because they are stored in ROM to look like descriptors.

The generation of each temporary results in inline constructor code which may bloat binaries where many string literals are used. Furthermore, the use of a run-time temporary is safe as long as it is used only during the lifetime of the function in which it is created

So, in summary, you should use _LIT to _L for your literal descriptors, because _L has an overhead associated with constructing a run-time temporary TPtrC and _LIT does not.

# posted by Jo Stichbury @ 5:36 PM 0 comments

22. What are literal descriptors?

Literal descriptors are constant descriptors which are compiled into ROM because their contents does not ever change. They are equivalent to static char[] in C.

There are two types of literal descriptor, _LIT which is the most efficient, and preferred, Symbian OS literal and _L, which is now deprecated in all but test code.

In the examples I've included in this blog, I've used _LIT throughout, so you've seen it in action already, for example:

_LIT(KFred, "Fred");

On the wide builds of Symbian OS created today, this is equivalent to the following:
_LIT16(KFred, "Fred");

If an explicitly narrow literal descriptor is required, this can be created using the _LIT8 macro:
_LIT8(KFred, "Fred");

KFred can then be used as a constant descriptor, either directly or to initialise a TPtrC/TPtr or TBufC/TBuf.

The macros are defined in the Symbian OS header file e32def.h and create an object of type TLitC16 or TLitC8 (defined in e32des.h). These classes do not derive from TDesC8 or TDesC16 but have the same binary layout as TBufC8 or TBufC16. This means that the literal object can be used wherever TDesC is used. The literal classes provide operator()() which is very useful to cast the literal object to a TDesC. For example:

_LIT(KFred, "Fred");
HBufC8* theHeapBuffer = KFred().AllocL();

To save you additional code, Symbian OS already defines a literal to represent a blank string, defined as follows:

Build independent: _LIT(KNullDesC,"");
8-bit for non-Unicode strings: _LIT8(KNullDesC8,"");
16-bit for Unicode strings: _LIT16(KNullDesC16,"");

# posted by Jo Stichbury @ 5:34 PM 1 comments

21. Why is there no HBuf class?

Well, there is and there isn't. Up until Symbian OS v8.0, there was no modifiable heap descriptor class, HBuf, in symmetry with the TBufC and TBuf stack buffers. There were a number of reasons for this:

(1) The expectation might be that, for a modifiable heap descriptor, the maximum length would expand and contract on the heap as the content was modified. An HBuf class which is modifiable but does not dynamically resize could be considered odd and rather pointless. But to add dynamic resizing to HBuf would be difficult because these methods could leave under low memory conditions and would have to be adjusted accordingly. But, as I’ve previously described, all the modifiable descriptor operations are implemented in the base class, TDes, thus affecting all descriptor types.

(2) If dynamic reallocation were added to HBuf, the location of the buffer might change as code is reallocated, affecting all TPtr objects referencing that heap buffer's data, as well as compromising the cleanup stack, if the HBuf pointer is stored there.

(3) Heap buffers were initially intended to allow efficient reading of constant resource strings of variable length. Because they are constant, they save on the additional four bytes required to store a maximum length, which allows them to be as compact as possible. They can still be modified using Des() if necessary, If a separate HBuf class were provided, it would have an extra 4-byte overhead - and may frequently be created unnecessarily, which might add to a significant waste of heap space over time.

However, this is a somewhat spurious argument since making an HBufC::Des() call is non-trivial and the resultant TPtr itself occupies 12 bytes of stack space. If a modifiable heap descriptor is definitely needed, creating a constant buffer and then an additional, TPtr to update wastes memory *and* processor instructions.

To this end, Symbian OS 8.0 and 8.1 introduced class RBuf for resizable descriptors. The Symbian documentation for this class can be found in [reference 6].

I discuss class RBuf in more detail in 32. How do I use RBuf? What is it?.

# posted by Jo Stichbury @ 4:53 PM 3 comments

20. How do I use heap-based buffer descriptors?

Heap-based descriptors, HBufC<n>, can be used for dynamic allocation of string data whose size is not known at compile time. It should also be used to store string data too large for the stack. In effect, HBufC should be used where malloc’d data would be used in C.

Although the class representing these descriptors is HBufC, these descriptors are always referred to by pointer, HBufC*. The class doesn’t comply with the standard Symbian OS naming conventions, but then, it doesn’t really fit any of the standard Symbian OS types exactly. It is simply prefixed with H to indicate that the data is stored on the heap.

HBufC can be created using one of the static NewL() functions of the class. These may leave if there is insufficient memory available (the non-leaving version will return NULL). The heap buffers must be constructed using one of these methods or by using one of the Alloc() or AllocL() methods of the TDesC class to spawn an HBufC copy of any descriptor.

The ‘C’ suffix in the class name indicates that these descriptors are constant, although, in common with the stack-based non-modifiable buffer descriptors, the class has a set of assignment operators to allow the entire contents of the buffer to be replaced (as long as the new data does not exceed the length of the heap cell originally allocated). In addition, as with TBufC, the heap-based descriptors can be manipulated at runtime by creating a modifiable pointer descriptor, TPtr over the heap buffer, using the Des() method. For example:

_LIT(KFred, "Fred");
// Allocate an empty heap descriptor of max length 4
HBufC* heapBuf = HBufC::NewL(4);
// Copy in the contents of KFred
*heapBuf = KFred; // heapBuf now contains "Fred"
// Or, a more direct alternative:
HBufC* heapBuf = KFred().AllocL();

_LIT(KBert, "Bert");
TPtr ptr(heapBuf->Des()); // Modifiable TPtr over heapBuf data area
ptr = KBert(); // Copies "Bert" into heapBuf
delete heapBuf; // heapBuf deleted - ptr is now invalid

The heap descriptors can be created dynamically to the size required, but are not automatically resized when additional data storage is needed. Instead, you must make sure the buffer has sufficient memory before calling a modification operation. If necessary, you can reallocate the buffer, to grow it to the desired size, using the set of ReAllocL() methods. These may potentially move the buffer from its previous memory location. If there is insufficient memory, they will leave.

_LIT(KFred, "Fred");
// Allocate a heap descriptor of max length 4
HBufC* heapBuf = KFred().AllocLC();

_LIT(KCyril, "Cyril");
TPtr ptr(heapBuf->Des()); // Modifiable TPtr over heapBuf data area
ptr = KCyril(); // This would panic because max length (4) is exceeded

// Instead, we need to do a reallocation
// Leave on cleanup stack in case the realloc fails and a leave occurs
heapBuf = heapBuf->ReAllocL(5);

// Realloc succeeded, but heapBuf pointer may have changed
// We must update the pointer stored on the cleanup stack
CleanupStack::Pop(); // Push it off
// Push it back on again
CleanupStack::PushL(heapBuf);

// Since realloc may have changed the location in memory
// we must also reset ptr
ptr.Set(heapBuf->Des());
ptr = KCyril(); // Copies "Cyril" into heapBuf

CleanupStack::PopAndDestroy(heapBuf); // heapBuf destroyed

Moral of the story
Heap descriptors can be created dynamically but are not automatically resized should they need to be extended.

# posted by Jo Stichbury @ 4:05 PM 7 comments

19. How do I use stack-based buffer descriptors?

The stack-based buffer descriptor classes are TBufC<n> and TBuf<n>. They are useful for fixed-size or relatively small strings, say up to the length of a 256-character filename (i.e. where n <=256). As I've previously described in 6. Large stack-based descriptors and in Tip 3, because they are stack-based, and stack space is limited on Symbian OS, if you need a larger buffer, you should use the heap (either by using the heap-based buffer descriptors, or having TBuf or TBufC as member variables of C classes). Being stack-based, these descriptors should also only be used when they have a lifetime that coincides with that of their creator.

The TBuf and TBufC descriptors are somewhat equivalent to char[] in C, but benefit from internal overflow checks. As you've probably guessed by now, they may be constant (TBufC) or modifiable (TBuf).

TBufC
The layout of the buffer descriptors is such that the string data forms part of the descriptor object itself, and follows the length word for TBufC and the maximum length word for TBuf. You can find a diagram in [reference 5].

TBufC is a thin template class which uses the TInt value to fix the size of the data area. The non-modifiable buffer class is used to hold constant string or binary data. It derives from TBufCBase (which derives from TDesC, and only exists as an inheritance convenience. It should not be used directly). You can find more about thin templates from Chapter 19 of my book, Symbian OS Explained, more details of which are in the References section.

TBufC defines constructors that allow it to be created from a copy of any other descriptor or from a zero-terminated string. They can also be created empty and filled later. This may at first seem unintuitive for a constant buffer, but in fact, while the data itself is constant, the contents of the buffer may be replaced by calling the assignment operator of the class. The replacing data must not exceed the length specified when the buffer was created, or a panic will occur, in both debug and release modes.

TBufC also has a Des() method which returns a modifiable pointer descriptor for the data represented by the buffer. So, while the contents cannot be altered directly by calling functions such as Format() or LowerCase() upon it, it is possible to change the data indirectly by creating a TPtr around the data in the buffer. When the modification occurs, the length of both the pointer descriptor and the constant buffer descriptor it refers to are changed. But remember, it cannot be extended because the descriptor classes do not provide memory management. So, don't call Append() on the TPtr unless you know there's sufficient space!

_LIT(KFred, "Fred");
_LIT(KBert, "Bert");
TBufC<4> fredBuf(KFred); // Constructed from literal descriptor
TPtr ptr(FredBuf.Des()); // max length = 4
ptr = KBert; // fredBuf now containts "Bert"

_LIT(KCyril, "Cyril");
ptr = KCyril; // Panic! KCyril exceeds max length of ptr (=4)

TBuf
Like TBufC, the corresponding modifiable buffer class, TBuf<n>, is a thin template class, where n defines the maximum length of the descriptor data area. TBuf derives from TBufBase, which itself derives from TDes. TBuf thus inherits the full range of descriptor operations from TDes and TDesC.

Although the buffer is modifiable, as with all descriptors, memory management of the TBuf is your responsibility and it cannot be extended beyond the initial maximum length set on construction. If it needs to expand, you need to make sure that you either make it large enough at compile time (but not too large to waste valuable stack space) or use a descriptor which you can dynamically allocate at runtime. This means using an HBufC heap descriptor, described in 20. How do I use heap-based buffer descriptors?.

# posted by Jo Stichbury @ 3:05 PM 6 comments

18. How do I declare a TPtr or TPtr8 for definition later?

I want to initialise a TPtr later, but declare it now. How do I do this?

TPtr8 myPtr(NULL, 0);

or, for member data, in the constructor's initialization list:

CMyClass::CMyClass()
: iPtr(NULL, 0)
{}

To assign to it later, use Set() as described in 16. What does TPtrC::Set() do?. For example:

myTPtr.Set(myHBufC->Des());
iPtr.Set(someExistingBuf);

# posted by Jo Stichbury @ 12:08 PM 0 comments

Saturday, May 14, 2005

17. What does TPtr::operator=() do?

The assignment operator, operator=(), defined for modifiable pointers only, copies data into the memory already referenced by the pointer.

If the length of the data to be copied exceeds the length available, the assignment will panic. TPtr also has a Set() method to change the descriptor to point at different data and you should be careful not to confuse them.

To reiterate:
As I described in 16. What does TPtrC::Set() do?, TPtr::Set() resets either a constant or modifiable pointer descriptor to point at a new data area (with corresponding changes to the length and maximum length members).

The assignment operator on TPtr copies data into the existing descriptor area if it is large enough, or panics if it is not. It may modify the descriptor length but will not affect its maximum length.

# posted by Jo Stichbury @ 2:00 PM 0 comments

16. What does TPtrC::Set() do?

Set() may be called on a constant or modifiable pointer descriptor to change what it points to, that is, to set it to reference different string data.

For example:

_LIT(KFred, "Fred");
_LIT(KBert, "Bert");
TPtrC fred(KFred); // contains 'Fred'
TPtrC bert(KBert); // contains 'Bert'
bert.Set(fred); // bert now contains 'Fred'

# posted by Jo Stichbury @ 1:59 PM 0 comments

15. What is the memory layout of pointer descriptors? How much memory do they occupy besides the data itself?

Since pointer descriptors all derive from TDesC (either directly for constant pointer descriptors or from TDes, for modifiable descriptors), the first 4 bytes are used to store the current length of the descriptor data.

In a constant pointer descriptor (TPtrC), a 4-byte pointer to the data follows the length word, thus the total size of the descriptor object is two words (8 bytes).

In a modifiable pointer descriptor (TPtr) which derives from TDes, the maximum data length word follows the current length, which is then followed by the data pointer. Thus, the descriptor object is three words in length (12 bytes).

You can find some diagrams which illustrate this in [reference 4].

# posted by Jo Stichbury @ 1:37 PM 0 comments

14. Do pointer descriptors own and manage the memory they point to?

No. Absolutely not.

The memory that holds the data referenced by a pointer descriptor is not owned by it, but will have been previously allocated. Pointer descriptors are agnostic about where the memory they point to is actually stored (in ROM, on the heap or on the stack).

The string data of a pointer descriptor is separate from the descriptor object itself and pointer descriptors themselves are usually stack-based, although they can be used on the heap, for example as a member variable of a CBase-derived class.

# posted by Jo Stichbury @ 1:18 PM 0 comments

Monday, May 09, 2005

12. What is the layout of modifiable descriptors?

Modifiable descriptors all derive from the base class TDes, which itself is a subclass of TDesC. TDes has an additional 4-byte member variable, which is used to store the maximum length of data currently allocated to the descriptor.

TDes defines a range of methods for modifying string data. Like the non-modifiable descriptors, all the manipulation is inherited by the derived classes, and works regardless of their type. The subclasses only implement specific methods for construction and copy assignment.

Apart from the allocation methods of the heap buffer descriptors, no descriptor functions allocate any memory. So any modification which extends the data of the descriptor, eg Append(), will check that there is sufficient memory available before proceeding. The contents of the descriptor can shrink and expand at will, as long as the length does not exceed the maximum. If the length of the descriptor contents is less than the maximum length, the end of the block of memory allocated to the descriptor is simply unused.

Descriptor code uses assertion statements to ensure that the maximum length of the descriptor is sufficient, before proceeding. The checks are made in both debug and release builds, and panic if an overflow would occur. This avoids the hard-to-trace memory scribbles and buffer overflows typical of C style strings.

(See also 11. What is the layout of non-modifiable descriptors?)

# posted by Jo Stichbury @ 4:27 PM 0 comments

11. What is the layout of non-modifable descriptors?

First, let’s distinguish between descriptor literals, which are constant and can be built into ROM because their contents is fixed at build time, and non-modifable descriptors, whose contents are constant but not fixed at build time. Symbian OS, literals are treated a bit differently to the other descriptors, and I’ll discuss them separately in 22. What are literal descriptors?.

All (non-literal) descriptors derive from the base class TDesC (typedef’d to TDesC16 in e32std.h and defined in e32des16.h). The ‘C’ at the end of the class name indicates that the descriptor is non-modifiable. TDesC class has methods for determining the length of the descriptor and accessing its data, and implements other descriptor methods which access the descriptor data but do not modify it, such as data access, comparison and search functionality.

It is worth pointing out that all non-modifiable descriptor methods are implemented in this generic base class rather than overridden by each non-modifiable descriptor subclass (TBufC, TPtrC and HBufC). This is for memory efficiency. It’s done this way, rather than by use of virtual functions, because these would require each derived descriptor object to occupy an extra 4 bytes by adding a virtual pointer (vptr) for access to the virtual function table. This memory size overhead is undesirable, particularly for smaller strings, where the 4 bytes becomes a large percentage of the overall size.

And by keeping the descriptor methods in the base classes, the amount of code to implement a full set of string functionality is minimized too. This reuse is a benefit in terms of ROM size, and an illustration of C++ best practice for testing and maintenance. The subclasses only implement specific methods for construction and copy assignment.

The first 4 bytes of every descriptor object is the same: these hold the length of the data it currently contains. Well, actually, only 28 of these 32 bits are used to hold the length of the descriptor data; the top 4 bits are used to indicate the type of descriptor. The use of 4 bits to identify the type limits the number of different types of descriptor to 2^4 (=16), but since only six types have been necessary in all previous releases of Symbian OS (TBufC, TBuf, TPtrC, TPtr, HBufC and RBuf) , it seems unlikely that the range will need to be extended significantly in future. The use of the other 28 bits to store the data length, also means that the maximum number of byes a descriptor may occupy is limited to 2^28 bytes = 256 MB.

The rest of the layout of a descriptor object depends upon the implementation of each of the subclasses. Access to the descriptor data of each type goes through the (nonvirtual) Ptr() method of the TDesC base class, which uses a switch statement to identify the type of descriptor (using the top 4 bits of the beginning of the descriptor object) and return the correct address for the beginning of its data.

Of course, this requires that the TDesC base class has knowledge of the memory layout of its subclasses hardcoded into Ptr(). This means that you can’t create your own descriptor class, deriving from TDesC, and expect it to work.

# posted by Jo Stichbury @ 3:28 PM 7 comments

10. How do I use heap descriptors as return types?

I want to create a new descriptor in my function. How do I return it to the caller?

You must return an HBufC* as follows:


HBufC* CreateSomeDescriptorL()
{
    _LIT(KBert, "bert");
    HBufC* newBert = KBert().AllocL();
    return (newBert);
}

The calling function needs to know that it must take ownership of the returned heap-based descriptor and be responsible for deleting it when it has finished with it. Failure to do this is a common cause of memory leaks.

A similar function, which leaves the created descriptor on the cleanup stack for the caller, would be coded as follows:



HBufC* CreateSomeDescriptorLC()
{
    _LIT(KBert, "bert");
    HBufC* newBert = KBert().AllocLC();
    return (newBert);
}

When should I return a TPtr or TPtrC? When shouldn't I?

TPtr or TPtrC are descriptors which do not own string data; they simply refer to data that exists in another descriptor. So you can use them to return a descriptor which references part of another descriptor argument, as long as its lifetime will not extend beyond that descriptor's lifetime. For example:


TPtrC LeftChar(const TDesC& aInput)
{
    if (aInput.Length()>0)
        return aInput.Left(1); // Returns the left-most character
    else
        return KNullDesC;
}

This, however, is not OK because stack-based fred will cease to exist when GetFred() returns:

TPtrC GetFred()
{  
  _LIT(KFred, "Fred");
  TBufC<4> fred(KFred());
  TPtrC fredPtr(fred);
  return (fredPtr);
}

# posted by Jo Stichbury @ 9:58 AM 5 comments

9. How do I use descriptors as parameters?

The descriptor base classes are described in 11. What is the layout of non-modifiable descriptors? and 12. What is the layout of non-modifiable descriptors?

The base classes provide and implement the constant and modifiable descriptor operations regardless of the actual type of the derived descriptor. For consistency, they should be used as arguments to functions, allowing descriptors to be passed without restricting the caller of the function to using a specific type.

For example, the caller can call the following function with anything derived from TDesC and TDes for the first and second arguments respectively. For example an HBufC and a TBuf<8>, or a TPtr and a TBuf<3>:

void SomeFunction(const TDesC& aReadOnlyDescriptor, TDes& aReadWriteDescriptor);

How do I make a descriptor parameter read-only?

Pass it as a constant reference to a non-modifiable descriptor (const TDesC&). For example:

void SomeFunction(const TDesC& aReadOnlyDescriptor);

This function can still be called with a modifiable descriptor like a TPtr as the argument, because these derive from TDesC (a TPtr is a specialised type of TDes). Inside the function, only the TDesC operations may be performed on the parameter. Its contents will not be changed, even if you have passed in a modifiable, TDes-derived descriptor.

How do I make a descriptor parameter read/write?

Pass it as a non-constant reference to a modifiable descriptor (TDes&). For example:

void SomeFunction(TDes& aReadWriteDescriptor);

You cannot call this function with a non-modifiable descriptor, which makes sense because you don’t want a read only descriptor to be modified.

Inside the function you can both read from and write to the parameter. Remember that the descriptor must already have sufficient space in it for the data to expand into, if necessary. If there isn’t, you’ll get a panic. (Some functions will be coded to check the length of the descriptor before attempting something that may cause a panic and return an overflow error instead. But others will not and the results will be messy).

[ This is one reason why you should always test your code with "boundary conditions" - that is, passing in parameters to functions at the boundary of their acceptable values, and beyond, to check that resulting errors are handled gracefully.]

Can I use other descriptor types as function parameters?

Yes you can, but don’t. For example you could specify your function requiring a parameter passed a TBuf<10>&. But what if the caller has got a TPtrC, or an HBufC, or even a TBuf<11>? This means that they won’t be able to call your function without some extra hassle to fit in with exactly what you have specified. TDes and TDesC are the most general types you can use for your function parameters.

My function uses TDes or TDesC parameters like you say, but my descriptor passing doesn’t work. Why not?

Oh how much pain has this one caused? Missing out that little & symbol makes all the difference. Your parameter types must be references, not values, ie const TDesC& or TDes&.

The base classes TDesC and TDes contain no string data. If you pass them by value rather than by reference, you are using static binding, which means that polymorphism won’t work, and you'll end up with a data-free base class object. It will all compile OK, but nothing works.

Never attempt to instantiate or work directly with objects of the base classes TDesC or TDes, as Tip 1 advises. They are effectively abstract classes. There is rarely, if ever, a valid reason for instantiating them rather than an object of their deriving classes (TBufC, TBuf, TPtrC, TPtr, HBufC or RBuf).

# posted by Jo Stichbury @ 9:36 AM 10 comments

Friday, May 06, 2005

8. So how do I create a heap based descriptor? And how do I use it when I've got it?

Create an HBufC (or an HBufC8 if you explicitly need 8-bit text).

HBufC* heapbasedFred=HBufC::NewL(4);

But how do I then write to a heap descriptor? HBufC is not derived from a modifiable descriptor class?

This is also easy. First you must call HBufC::Des() to get a TPtr, which is modifiable. You can then use that:

TPtr fredPtr(heapbasedFred->Des());
_LIT(KFred, “fred”);
fredPtr.Copy(KFred);

And how do I read from HBufC?
You can just dereference the HBufC pointer, like this:

TBuf<4> stackbasedFred(*heapBasedFred);

But I’ve seen code which calls Des() on an HBufC. Why’s that?

This is a common mistake, which fortunately does little harm except to overall efficiency.
Des() gives you a modifiable descriptor, TDes, which itself derives from TDesC, so you can use it to call any of the TDesC functions. But it’s unnecessary.

Here’s an example of this common mistake:

TBuf<4> stackbasedFred(heapBasedFred->Des()); // Unnecessary, just use
TBuf<4> stackBasedFred(*heapBasedFred); // More efficient

# posted by Jo Stichbury @ 2:02 PM 2 comments

7. How do I create a small, stack-based descriptor?

If it needs to be modifiable, use a TBuf. If it's constant, use a TBufC. You'll need to specify the size of stack space allocated for the descriptor's data.

Here’s an example:

_LIT(KFred, “fred”); // A string literal, these will be described later
TBufC<4> constantFred(KFred);

In reality, you probably wouldn't do this for a non-modifiable descriptor, since you can use the literal, KFred directly by calling the operator()() (as I'll describe separately in 22. What are literal descriptors?). That is, instead of creating constantFred, you could just call KFred().

However, this approach is still useful for modifiable descriptors, for example, for logging purposes:

TInt CMyActiveObject::RunError(TInt aError)
{
_LIT8(KError, "CMyActiveObject::RunError: %d");
TBuf8<35> errorBuf;
errorBuf.Format(KError, aError); // reference 3 has more information about TDes::Format()
iLogger.Log(errorBuf); // iLogger is a log object owned by the class
return (KErrNone);
}

What if I want a descriptor to hold ASCII text?

There are some cases where you do need to hold 8-bit text: MIME types, or binary data, for example. To do this, you should explicitly use the 8 bit descriptors:

_LIT8(KImageGif, “image/gif”);
TBufC8<15> mimeType(KImageGif);

# posted by Jo Stichbury @ 2:01 PM 2 comments

Thursday, May 05, 2005

6. Large stack-based descriptors

The amount of stack space on Symbian OS is pretty limited (the default is 8 KB). So you should avoid creating large stack-based descriptors when it is unnecessary to do so. Symbian OS sometimes makes it easy to transgress this rule, by defining a number of classes and typedefs that can be used inefficiently on the stack. It pays to be aware of the these and only use them when you know the descriptor you are reading will fill the entire space allocated.

For example, TFileName is typedef-ed as follows:

typedef TBuf<KMaxFileName> TFileName;

where

const TInt KMaxFileName=0x100; // = 256 decimal

But, of course, each character in a descriptor is 2 bytes, since Symbian OS is a wide, UNICODE, build (as described in 1. The Basics). So each TFileName object created on the stack occupies 512 bytes (1/16th of the default stack size), regardless of whether the text occupying it is actually that long!

Sure, these objects can be very appealing, because they mean you don't have to worry about buffer overrun. But they come at a price.

For stack conservation, it's advisable to be aware of the amount of space the following objects consume:

TFileName 512 bytes
TEntry 544 bytes
TFullName 512 bytes
TName 256 bytes

If you do need to use these objects, it's best to use them on the heap. You can do this simply by making them a member of a heap-based object (ie a C class object) . They don't then need to be created on the heap themselves - just by being a member of the C class, they are automatically heap-based. Alternatively, you can alloc them on the heap using the new operator - but make sure they are leave-safe and destroyed when no longer needed.

# posted by Jo Stichbury @ 11:17 AM 4 comments

5. Is there any size limit for a descriptor?

Sort of.

Descriptors can be stored on the stack or heap. The normal limitations apply as for other Symbian OS variables:

Stack descriptors should be kept small: a 256 byte limit (128 double-byte characters) is a good guide. See 6. Large stack-based descriptors for more information about how to avoid accidentally wasting stack space through use of Symbian OS descriptor typedefs and stack-based classes which encapsulate large descriptors.
Heap descriptors can be much larger, depending on the size of heap memory available. But remember, Symbian OS runs on limited memory devices, so allocating a massive descriptor is always unwise.

But the layout of a descriptor object (described elsewhere) will limit the maximum size of a descriptor to 2^28 bytes (256 MB). Since each UNICODE character = 2 bytes, the maximum length of a descriptor is thus 2^27 characters.

# posted by Jo Stichbury @ 11:10 AM 2 comments

4. Memory management and descriptors

Do descriptors automatically resize themselves if I need more memory?
No. The memory allocated for a descriptor is fixed, and specified on construction. They do not resize themselves dynamically, although you can re-allocate memory for an HBufC heap descriptor. But this has pitfalls of its own as will be described in 20. How do I use a heap-based buffer descriptor?

Why? It would be more useful if it were automatic.
It would make life easier for the programmer, but it could also make them less efficient, and it would require complex code to be exception-safe. Think of it like this:

Most descriptor modification methods can potentially require an increase in the amount of memory allocated to the descriptor. If the descriptor classes themselves did this, most of the methods would have to be able to leave, or would be surrounded by less-than-efficient TRAPs. This is quite an overhead when, in the majority of cases, a fixed size string is fine.

On Symbian OS, efficiency is paramount, so the programmer must take responsibility for memory management.

What happens if I overrun a descriptor’s allocated space?
You will get a panic in both debug and release builds. The panic will be part of the USER category which can be found documented in [reference 2]. It can be assumed that no illegal access of memory has taken place and that no data was moved or corrupted. However, your code will rapidly cease executing, and if you are running in a system thread, you could even reboot the phone.

Isn’t that a bit drastic?
Drastic maybe, but safe. You will not be overwriting any memory that you shouldn’t be, which avoids the difficult-to-find errors this can cause for C strings.

Do descriptors perform garbage collection?
No. In the same way that descriptors do not perform allocation or re-allocation, they do not perform garbage collection, because of the extra overhead that would carry.

# posted by Jo Stichbury @ 10:57 AM 1 comments

Wednesday, May 04, 2005

3. What aren't descriptors?

Because their underlying memory allocation and cleanup must be managed by the programmer, as described in 4. Memory management and descriptors, they’re not like standard C++ strings, Java strings or the MFC CString.

And because they protect against buffer overrun and don’t rely on NULL terminators to determine the length of the string, they are not like C strings either. See 2. Why use descriptors? for why you should prefer to use descriptors rather than familiar, but clunky, C strings.

# posted by Jo Stichbury @ 3:04 PM 0 comments

2. Why use descriptors?

Can I use standard C arrays to store my text and binary data, and the C string library to manipulate them, like I have always done?
Yes.

Should I use standard C arrays and the C string library for Symbian OS programming?
No.

Why not?
Standard C arrays are unsafe. Why? Because they have no concept of how big they are. Null-terminated strings are clunky and inefficient. What’s more, if you want to write past the end of the allocated space of a standard C string, nothing is going to stop you. This leads to all kinds of mayhem, such as data corruption

An API I want to use requires descriptors, but I’ve used C strings. How do I convert between them?
Quite. Just about every Symbian OS C++ API you come across will use descriptors for transferring text and binary data somewhere, so if you’ve used C strings, you are going to have a problem using these APIs. You’ll have to do the conversion every time you want to use an API call requiring a descriptor, which is inefficient. So you might as well keep reading, and learn how to use descriptors correctly.

# posted by Jo Stichbury @ 12:58 PM 3 comments

1. The Basics

What are descriptors?
Descriptors are Symbian OS strings.

So why are they called 'descriptors' rather than strings?
They're known as 'descriptors', because they are self describing. Each descriptor object holds the length of the string of data it represents as well as its 'type', which identifies the underlying memory layout of the data it holds.

So they can be used to store text?
They can. They can also be used to store binary data.

Do they store text data as ASCII or Unicode?
Either. There are two types, 8-bit and 16-bit descriptors. The 8-bit variant is used to store ASCII characters and raw binary data. The smallest data unit in these is a “narrow” 8 bit (1 byte) character. The 16-bit type is used to store Unicode characters; for these, the data unit is a “wide” 16-bit (2 byte) character.

How do I know which I have got?
Each type ends in 8 or 16, for example TDesC8 or TDesC16.

What about descriptors without 8 or 16 at the end, such as TDesC?
These are said to be of ‘neutral’ width because they are built with the platform default character size. In the old days, Symbian OS (or EPOC32 as it was known then) used 8-bit ASCII text, so a neutral descriptor defaulted to the 8-bit variety. Since Symbian OS v5U (also known as ER5U) Symbian OS has used wide strings.

Thus, unless you're working with a pre-v5U SDK, the neutral descriptors (TBufC, TPtr, HBufC etc) are implicitly 16-bit and are actually TBufC16, TPtr16, HBufC16 etc.

How do I find out the character width of my platform?
As described above - unless you’re working with a version of the OS which pre-dates Symbian OS v5U (used in the Ericsson R380 mobile phone in the year 2000) you’ll be working with the wide, 16-bit platform.

So which width should I use?
If you need to use 8-bit data, for example, to call a particular API method (such as the Read() or Write() functions of RFile), for Internet email, or to manipulate ASCII data, you should use the explicitly 8-bit descriptors.
If you want to store a chunk of binary data, use the 8 bit variety too.

If you need to work with 16-bit text, for example, Java strings, and want to indicate this explicitly, use the 16-bit descriptors.

But, in general, unless you need to make the data width explicit, use the neutral descriptors. If nothing else, your code is more readable and it may even be portable if the default character width changes on a later platform.

Are they NULL-terminated like C strings?
No.

How come?
Because they're self-describing, descriptors know the length of their internal data. This means there’s no need for the data to be NULL-terminated to mark where the data ends. It also means that binary data can be stored in the same descriptor classes as store text data. This isn’t possible with C strings because there’s no way of distinguishing the NULL terminating character from valid binary data.

Are they hard to use?
Not really. They’re easy to use badly but in fact there are only a few rules, and a few gotchas. They just haven’t been fully explained: until now...

You can find an overview of descriptors in the Symbian Developer Library which accompanies each Software Development Kit (e.g. for UIQ or S60 phones). It is also available online as described in the references section (References to other descriptors resources).

The descriptors overview can be found in [reference 1].

# posted by Jo Stichbury @ 12:37 PM 2 comments

A Different Approach to Explaining Descriptors

Every Symbian OS programming book has a chapter on descriptors, which is usually fairly near the start. They are all quite similar and will show the class hierarchy, a few diagrams with boxes and pointers, and more often than not will give some comparison with an equivalent C string type for each descriptor type. Then there will be a few examples of how to use them, which will all work.

You will read this not-too-interesting chapter while trying to keep awake; you’ll look at the examples and it will all make sense so you’ll think you can use descriptors. You may still have a few worries in your mind, like “Why doesn’t Ptr() return a TPtr?”, but you move on. Then you do some development for real, which inevitably involves descriptors, and bang. It all goes wrong. You can’t decide which descriptor type to use and, when you think you’ve got it, the darned things won’t compile, or work without crashing or simply work at all. But don’t worry, this happens to many new Symbian OS developers, if not all of them. We’re here to help by taking a new approach to explaining descriptors.

FAQs Facts, Fax
The blog will consist of frequently asked questions. It’s difficult to take in information from a long, dry and dull chapter on descriptors. FAQs are different. They are much more interesting to read and present one piece of information at a time.

Good examples, bad examples
Symbian OS books always give examples that work (or that should work but for bugs introduced during publication). We are usually left to discover how to misuse descriptors at our leisure. Then we have to figure out why they are not working. This is hardly leisure. So this article will give lots of misuse examples too. We’ve made all our mistakes public so you don’t have to make them too.

Repetition
We say the same things again and again in different ways. We repeat ourselves. This is intentional. Some of these things are so important that you must understand them in order to use descriptors successfully. If repeating and re-describing helps your understanding, it's worth the effort.

Did we mention that we repeat ourselves?

# posted by Jo Stichbury @ 11:04 AM 3 comments

Welcome to the Descriptor FAQ

Someone famous once said that some high percentage of computing was manipulating strings. If you believe this to be true, then getting the hang of string handling is vital.

Most programmers come to Symbian OS from some other development environment, and tend to think they’ve already got strings pretty much sussed. After all, there’s not much to C strings to understand, although there are plenty of ways they can go wrong. The Java String and StringBuf classes have about the best combination of power and simplicity that you can get. And the various String and CString classes found in different flavours of C++ environments are usually mastered fairly quickly.

But then they encounter Symbian OS descriptors. If there was anything invented to bring a high flying C++ developer firmly back to ground during their first week of Symbian OS development, it’s descriptors. “Good grief, there’s a whole zoo of them” is a typical comment upon the first reading of the descriptor chapter in a Symbian OS programming book. They look at the various diagrams showing the C equivalents for each descriptor types. They try the examples. It seems just about to make sense. And then they try using them for real, and it all goes wrong.

Descriptors seem designed to restrict you. They don’t compile when you think they should. Or they compile without complaint, but panic when you use them. And if they do compile and run without crashing, they still may just plain refuse to do what you expect them to do. At which point it’s tempting to turn to the chapter in the book on using the standard C library and use C strings instead.

Well don’t. Stick with them. You don’t really have a choice anyway if you are going to be a Symbian OS developer. There are many good reasons to use descriptors, as will become apparent as we proceed.

What is needed is a really good explanation of descriptors in simple language that non-Symbian OS developers can understand. It should be detailed, because without the detail you can’t use them successfully. There should be examples of how not to use them, as well as the more typical book examples of how they should be used.

All Symbian OS developers will have made the same mistakes of using descriptors incorrectly and not understanding what’s going wrong. This article sets out to short-cut the painful learning curve. The authors have been along it, slowly and painfully. Don’t follow in their footsteps.

# posted by Jo Stichbury @ 10:23 AM 1 comments

I've got a question or comment not already listed. What do I do?

Let me know by adding it as a comment to the most relevant post in the blog (I get notification when a comment is added). Please feel free to add as many questions as you like, though bear in mind it may take me a while to get around to answering them. I'd like this to be the most comprehensive online resource for Symbian OS descriptors available, so your input is really valuable. If you have any descriptor tips you'd like to share, for the descriptor tips blog (descriptor-tips.blogspot.com), please add them as comments to the blog and I'll transfer them - and attribute them to you of course!

Alternatively, if you have a comment to a particular item, you can post it directly by clicking on the comments link at the bottom of the posting. I'll regularly monitor the comments and make corrections/additions accordingly.

# posted by Jo Stichbury @ 10:15 AM 11 comments