Tips for the Rogue Archer in Dragon Age Origins?

Agree with the recommendation to check out the guide on the wiki, it is pretty good.

Here are some thoughts:

First off Archers are really more party support characters than damage characters in DAO (until Awakening anyway).

Second thing, most of the Archery active talents are not worth using, but a couple are very good. Scattershot is excellent, and Shattering Shot is pretty good. Arrow of Slaying's damage calculation is bugged in the base game, and even with that fixed it isn't that great in my opinion. Pinning Shot is not that great, but maybe use it once or twice early. Critical Shot and Crippling Shot are not worth it most of the time.

The sustained talents are better, with most of them useful depending on the situation. If you are going Cunning and building for ranged crit chance, then you'll want to run Aim after you get halfway decent ranged crit chance, and also ideally gear with "Rapid Aim" property. Early when crit chance is low, run Rapid Shot for higher overall dps (even without crits). Defensive Fire can be handy early if you have to tank (admittedly more likely in a solo than a party game), since it gives 30 Defense if you also have Master Archer.

Suppressing Fire is a nice party support sustain to debuff enemies, and it can be combined with those other modes.

Stat wise, you need 34 dex (after bonuses) for the best bows, then you can put the rest into Cunning*. While you in theory need 20 strength to equip the best light armor, you can use +Strength equipment to meet that requirement.

If you are going the crit build, all your gear should be focused around accumulating Ranged Critical Chance, or Rapid Aim. Repeater Gloves from Return to Ostagar have the biggest bonus to aim time (probably an error).

Bard is the best specialization for archer since Song of Courage is one of the best rogue abilities, and you'll need it to buff your crit chance, Attack rating, and damage. Songs of Courage do stack, although activating it drops the other instances. So if I, Leliana and Zev activate it in that order, I get 3x stack, Leliana gets 2x, and Zev just gets his.

The other specialization is a bit trickier. Ranger is fine if you want pets. Assassin is good for Mark of Death and that is it. Duelist is not horrible since you will have issues with Attack rating early on, and Dueling can be applied to archery for +10 attack (probably an oversight, oh well).

Build order wise, would try to get Master Archer and Shattering Shot by Level 5 / Joining. Pick up Scattershot around Level 12 (should have enough Dex and Master Combat Training).

Loose ends:

Like with 2H warrior (and most any class anyway), you can animation cancel to get faster attack speed. After the arrow is shot, tap forward and then target the enemy again to skip the last part of the shot animation. Helps with DPS a little.

Although Cunning archer will do higher dps compared with Dex archer, the difference in on target dps isn't game breaking given the difference in attack rating (Cunning archer does more damage per hit, but Dex archer more likely to hit). Also Dex archer is stronger in Awakening by a lot due to the new sustain. The point is that although above (*) I said 34 dex, rest into cunning, like any rogue as long as your are investing in either dex or cunning the whole way, the character will still be pretty good.