How to use the record button in AppleScript

try this little syntax change:

on setfmFileOptions(app_name)

    -- choose the Developer Utilities menu item from the Tools menu of FileMaker Pro Advanced

    do_menu(app_name, "File", "File Options...")


    -- click the "Log in using" checkbox:
    tell application "System Events"
        tell process app_name
            repeat until (exists checkbox "Log in using:" of tab group 1 of window 1)
                delay 1
            end repeat
            click checkbox "Log in using:" of tab group 1 of window 1
        end tell
    end tell
end setfmFileOptions

You can't use window 1 without telling what process window 1 is in. You'll have to specify app_name as a parameter now.

So, like this:

setfmFileOptions("FileMaker Pro Advanced")

You can use a try block like this:

repeat -- your loop
    try
        --
        -- your code in the loop
        --
    on error err number n
        if n = -128 then -- User canceled
            --
            -- code to  cleanup before the program closes
            --
            return -- exit this script
        end if
    end try
end repeat