Mary Rodgers, who has died aged 83, was a composer and songwriter whose 1959 musical Once Upon a Mattress, based on a Hans Christian Andersen story, The Princess and the Pea, is played constantly throughout the US. She was part of a line of musical theatre composers: Richard Rodgers, of Rodgers and Hammerstein, was her father, and her son Adam Guettel is now in the business.

She never repeated the early success of Once Upon a Mattress, but wrote many songs with her great friend Stephen Sondheim, with whom she more or less grew up, before switching to children's fiction and writing a bestseller, Freaky Friday (1972), which spawned two body-swap movies in which mother and daughter trade places. The roles were played in 1976 by Barbara Harris and Jodie Foster, and in 2003 by Jamie Lee Curtis and Lindsay Lohan.

Rodgers was a remarkable woman in many ways, at once vivacious and mild-mannered, with a touching honesty about her own limitations. "I had a pleasant talent," she said, "but not an incredible talent. I was not my father or my son. And you have to abandon all kinds of things."

After her father died in 1979, she wrote a new introduction to his autobiography, in which she hinted at the dark side of life with him. He had been revealed as a serial philanderer and depressive alcoholic; when commiserated on the loss she must have felt, she said she had replied to correspondents: "I didn't lose any more than you did, folks, because basically what I loved about him was only what you loved – his music."

Richard Rodgers himself, though, gave the appearance of being besotted with his first daughter, relishing the story about Mary reporting the arrival of her younger sister in 1935: "She announced to her class, while preparing to appear in a school play, that her mother had just had a baby and that she couldn't possibly come to see the play because she was getting married."

Mary was closer to her mother, Dorothy (nee Feiner), collaborating with her on several projects in later life, including a book, A Word to the Wives (1970), a syndicated radio programme and a monthly column for McCall's magazine.

She was born in Manhattan, New York, was educated at Brearley school and, to the surprise of her father, majored in music at Wellesley College, Massachusetts. She grew up through the years of his greatest success: by the time she was 18, Richard Rodgers had written Pal Joey and The Boys from Syracuse with Lorenz Hart, and Carousel and South Pacific with Oscar Hammerstein.

When Once Upon a Mattress (with lyrics by Marshall Barer and a book by Jay Thompson and Dean Fuller) opened on Broadway, directed by George Abbott at the St James theatre, Brooks Atkinson of the New York Times hailed a genuine new composer – her music was "nothing like dad's," he said – and a funny new clown, Carol Burnett, who became an overnight star.

Although the show opened in the same season as her father's Flower Drum Song and Jule Styne and Sondheim's Gypsy, it still ran for a year, with Burnett knocking 'em dead as the comically uncouth Princess Winnifred (Prince Dauntless had a song called "I'm in love with a girl named Fred"), and Buster Keaton joining the cast as the emaciated King Sextimus on its hops between other Broadway theatres.

That latter role was taken in London by Max Wall, but the 1960 Adelphi production lasted for only 38 performances. A small-scale 1985 revival by Wendy Toye at the Watermill, Newbury, however, was a delight, revealing a score full of beguiling romantic ballads and feisty comic numbers; the cast of unknowns included Sally Dexter, Douglas Hodge and Charles Lewsen (a sometime critic on the Times) as Sextimus.

A revue of Mary's songs, From A to Z, closed in a month in 1960, and her next musical, Hot Spot (1963), for which Sondheim rewrote the opening number with the lyricist Martin Charnin, is remembered only for being the last Broadway show featuring Judy Holliday. The critic John McLain wrote: "It is difficult to conceive a property which can smother the prodigious and ebullient Miss Holliday, but I'm afraid Hot Spot gets the job done."

She had more success with a collaborative revue inspired by Mad magazine, The Mad Show (1966), co-authoring with Sondheim a parody of the bossa nova hit The Girl from Ipanema, and changing her sex ("Why are his trousers vermilion? Why does he claim he's Castilian? Why do his friends call him 'Lillian'?"), an item that featured in the London revue Side by Side by Sondheim (compered by Ned Sherrin, produced by Cameron Mackintosh) that really rubber-stamped Sondheim's standing in Britain in 1976.

Other Broadway work included The Mad Woman of Central Park West (1979), a solo musical show for Phyllis Newman to which she contributed alongside Leonard Bernstein, Sondheim, Charnin, Barry Manilow, John Kander and Fred Ebb (it ran for three months only) and a compilation cabaret of her songs by Richard Maltby Jr, Hey, Love (1993).

Her books included two more identity-switch stories, A Billion for Boris (1974) and Summer Switch (1982). She also wrote the screenplay for a moderately funny Faustian-pact movie, The Devil and Max Devlin (1981), starring Elliott Gould and Bill Cosby.

Rodgers remained active in show business, serving on the boards of the Juilliard School and Lincoln Centre. She is survived by three children, Tod, Kim and Nina, from her first marriage, to the lawyer Julian Beaty (they divorced in 1957); by two sons, Alec and Adam, from her second marriage to the movie executive Henry Guettel, who died last year; and by seven grandchildren.

• Mary Rodgers, composer and writer, born 11 January 1931; died 26 June 2014