In their first tête-à-tête in nine months, the two South Asian leaders said they were in favour of renewed peace talks. The two men described the discussions as positive and mandated their respective foreign ministers to draw up a road map for future talks.
Although both sides appear in favour of renewed talks, they were still apart over differences that emerged following the Mumbai attacks.
Nonetheless, officials are set to work out ways to restore trust and confidence, "thus paving the way for a substantive dialogue on all issues of mutual concern", Indian Foreign Secretary Nirupama Rao said.
Pakistan's Foreign Minister Shah Mehmood Qureshi hailed the fact that future talks would go beyond tackling militancy and would address other areas of dispute.
During the talks with Gilani, Rao said Singh was "very emphatic that Pakistan has to act, that the terror machine needs to be controlled, needs to be eliminated".
Gilani responded by stressing that Pakistan was as much a victim of terrorism as its neighbour.
No timetable for the two foreign ministers to meet was agreed, but their get-together is expected "as soon as possible".
Singh also accepted an invitation by Gilani to visit Pakistan. The last Indian premier to make an official visit to Pakistan was Atal Bihari Vajpayee in 1999.
India blames Lashkar-e-Taiba, an Islamic extremist group, for the Mumbai attacks that killed about 170 people. Delhi also blames Islamabad for allowing extremists to have bases on its territory and for failing to find and arrest the organisers of the attacks.
Pakistan has admitted that the terrorist command that attacked Mumbai came from its territory but denied any involvement in the affair.
In February, Pakistan and India held their first formal talks since the 2008 attacks and agreed to "remain in touch". The two sides, however, made no substantial progress.
A few days ago, India arrested a woman working as a diplomat in its Islamabad embassy on charges of spying for Pakistan.
Since independence and partition in 1947, the two neighbours have had tense relations and fought three wars.
Experts view positively the recent rapprochement between the two powers, but agree that it will take time to build mutual trust.