Foundation Models Tools not invoking

The models decide if and when to call a tool based on the tool's description and if the generation process needs the tool. I'd suggest that you try to adjust your instructions / prompt / tool description to be clear when the models should call the tool and for what purpose. This Apple sample demonstrates a workable tool calling, and so you can probably start from there.

If that doesn't help, and you don't mind to share your project that reproduces the issue, I'd be super curious to take a look.

Best,
——
Ziqiao Chen
 Worldwide Developer Relations.

What am I missing here?

struct ContactsTool: Tool {

    let name = "getContacts"

    let description = """
        Get a contact from the address book matching the name
        """

    @Generable
    struct Arguments {
        
        @Guide(description: "The search query used to search the contact example name or email or contact number")
        let searchQuery: String?
    }

    func call(arguments: Arguments) async -> ToolOutput {
        do {
            // Request permission to access the person's contacts.
            let store = CNContactStore()
            try await store.requestAccess(for: .contacts)

            let keysToFetch = [CNContactGivenNameKey, CNContactBirthdayKey] as [CNKeyDescriptor]
            let request = CNContactFetchRequest(keysToFetch: keysToFetch)

            var contacts: [CNContact] = []
            try store.enumerateContacts(with: request) { contact, stop in
               
                if contact.givenName.lowercased().contains(arguments.searchQuery?.lowercased() ?? "") {
                    contacts.append(contact)
                } else if contact.middleName.lowercased().contains(arguments.searchQuery?.lowercased() ?? "") {
                    contacts.append(contact)
                } else if contact.familyName.lowercased().contains(arguments.searchQuery?.lowercased() ?? "") {
                    contacts.append(contact)
                } else if contact.emailAddresses.isEmpty == false {
                    for email in contact.emailAddresses {
                        if email.value.lowercased.contains(arguments.searchQuery?.lowercased() ?? "") {
                        contacts.append(contact)
                        break
                      }
                    }
                }
            }
            guard let pickedContact = contacts.shuffled().first else {
                return ToolOutput("No contact found")
            }
            return ToolOutput("These are the picked contact from your phone: name :\(pickedContact.givenName) \(pickedContact.familyName) email: \(pickedContact.emailAddresses.first?.value ?? "")")
        } catch {
            return ToolOutput("No contact found")
        }
    }
}


struct DocumentGenerationAssistTool: Tool {
    var name: String = "documentGenerationAssistTool"
    
    var description: String = "provides sample data for document generation"
    
    @Generable
    struct Arguments {
        
        @Guide(description: "the name or type of the document.")
        var documentName: String
    }
    
    
    func call(arguments: Arguments) async throws -> ToolOutput {
        return ToolOutput(GeneratedContent(SignRequest.sampleNonDisclosureAgreement.documet))
    }
}```


My session Creation:

```language
  static let modelInstruction = Instructions {
        "You are a document generation assistant for an eSignature platform. Your task is to convert plain text input into professional, well-structured HTML documents that will be used in legally binding signing requests. These documents will be edited in a rich text editor and exported to PDF."
        
        "The user may only specify the document name and may be a list of signers or receiver or approver"
        
        """
        Always use the \(contactTool.name) tool to find the recipients
        """
        
        """
        Always use the \(documentAssistTool.name) tool for the document generation
        """
    }
    
    private var session = LanguageModelSession(tools: [contactTool, documentAssistTool], instructions: modelInstruction)```

My user prompt -> "Generate a rental agreement with minimum of 2 pages need to be send for baarathi as a signer residing in Guduvancherry chennai pincode 603202"

I tried your instructions and prompt and the models said it couldn't assist with generating legal documents. I tried completely different instructions and prompt and did see that the tools were called. (My input was far from asking the models to generate a rental agreement, and I won't bother to elaborate.)

That being said, it's not that your tools have anything wrong; it is that the models reject your request in the first place. Splitting the task into smaller ones (for example, generating a section, rather than a whole document), providing a sample output, or engineering your prompts in other ways may help, but that will be up to you to experiment.

Should the models be able to full your current instructions and prompt? I don't have a definitive answer, and would suggest that you file a feedback report for the framework team to evaluate – If you do so, please share your report ID here for folks to track.

When you file feedback report, it’s super important to add the language model feedback attachment, which contains the session transcript that helps us reason the model’s output and analyze the error.

Best,
——
Ziqiao Chen
 Worldwide Developer Relations.

However, I’ve noticed that my model occasionally triggers the contact tool when I regenerate a response within the same session, even without changing the prompt. Also, if you don’t mind, could you please share the instruction and prompt you used?