1// Copyright (C) 2022 Intel Corporation.
2// SPDX-License-Identifier: LicenseRef-Qt-Commercial OR GFDL-1.3-no-invariants-only
6 \title Inter-Process Communication
8 \ingroup frameworks-technologies
10 \ingroup explanations-networkingandconnectivity
12 \brief An overview of Qt's inter-process communication functionality
14 Qt supports many ways of communicating with other processes running in the
15 same system or in different systems. There are basically three types of
16 inter-process communication mechanisms:
19 \li Synchronization primitives
20 \li Exchanging of arbitrary byte-level data
21 \li Passing structured messages
24 \section1 Synchronization primitives
26 Qt only provides one class for explicit inter-process synchronization:
27 \l{QSystemSemaphore}. A QSystemSemaphore is like a \l{QSemaphore} that is
28 accessible by multiple processes in the same system. It is globally
29 identified by a "key", which in Qt is represented by the \l{QNativeIpcKey}
30 class. Additionally, depending on the OS, Qt may support multiple different
31 backends for sharing memory; see the \l{Native IPC Keys} documentation for
32 more information and limitations.
34 It is possible to use regular thread-synchronization primitives such as
35 mutexes, wait conditions, and read-write locks, located in memory that is
36 shared between processes. Qt does not provide any class to support this,
37 but applications can use low-level operations on certain operating systems.
39 Other Qt classes may be used to provide higher-level locking, like
40 \l{QLockFile}, or by acquiring a unique, system-wide resource. Such
41 techniques include \l{QTcpServer}{TCP} or \l{QUdpSocket}{UDP} ports or
42 well-known names in \l{QDBusConnection::registerService}{D-Bus}.
44 \section1 Byte-level data sharing
46 Using byte-level data, applications can implement any communication
47 protocol they may choose. Sharing of byte data can be stream-oriented
48 (serialized) or can allow random access (a similar condition to
49 QFileDevice::isSequential()).
51 For serial communication, Qt provides a number of different classes and
54 \li Pipes and FIFOs: \l QFile
55 \li Child processes: \l QProcess
56 \li Sockets: \l QTcpSocket, \l QUdpSocket (in \l{Qt Network})
57 \li HTTP(S): \l QNetworkAccessManager (in \l{Qt Network}) and
58 \l QHttpServer (in \l{Qt HTTP Server})
59 \li CoAP(S): \l QCoapClient (in \l{Qt CoAP})
62 For random-access data sharing within the same system, Qt provides
63 \l{QSharedMemory}. See the \l{Shared Memory} documentation for detailed
66 \section1 Structured message passing
68 Qt also provides a number of techniques to exchange structured messages
69 with other processes. Applications can build on top of the byte-level
70 solutions above, such as by using \l QJsonDocument or \l QXmlStreamReader /
71 \l QXmlStreamWriter over HTTP to perform JSONRPC or XMLRPC, respectively,
72 or \l QCborValue with QtCoAP.
74 Dedicated Qt modules for structured messages and remote procedure-calling
78 \li \l{Qt Remote Objects}
84 \page shared-memory.html
87 \keyword shared memory
89 \brief Overview of the techniques for sharing memory between processes.
91 Qt provides two techniques to share memory with other processes in the same
92 system: \l{QSharedMemory} and memory-mapped files using \l{QFile}. Memory
93 that is shared with other processes is often referred to as a "segment",
94 and although it may have been implemented as specific segments on
95 processors with segmented memory models in the past, this is not the case
96 in any modern operating system. Shared memory segments are simply regions
97 of memory that the operating system will ensure are available to all
98 processes participating.
100 \note The address at which the segment is located in memory will almost
101 always be different for each process that is participating in the sharing.
102 Therefore, applications must take care to share only position-independent
103 data, such as primitive C++ types or arrays of such types.
105 \section1 Sharing memory using QSharedMemory
107 QSharedMemory provides a simple API to create a shared memory segment of a
108 given size or attach to one that was created by another process.
109 Additionally, it provides a pair of methods to \l{QSharedMemory::}{lock}
110 and \l{QSharedMemory::}{unlock} the whole segment, using an internal
111 \l{QSystemSemaphore}.
113 Shared memory segments and system semaphores are globally identified in the
114 system through a "key", which in Qt is represented by the \l{QNativeIpcKey}
115 class. Additionally, depending on the OS, Qt may support multiple different
116 backends for sharing memory; see the \l{Native IPC Keys} documentation for
117 more information and limitations.
119 QSharedMemory is designed to share memory only within the same privilege
120 level (that is, not with untrusted other processes, such as those started
121 by other users). For backends that support it, QSharedMemory will create
122 segments such that only processes with the same privilege level can attach.
124 \section1 Sharing memory via memory-mapped files
126 Most files can be mapped to memory using QFile::map() and, if the
127 \l{QFileDevice::MapPrivateOption}{MapPrivateOption} option is not specified,
128 any writes to the mapped segment will be observed by all other processes
129 that have mapped the same file. Exceptions to files that can be mapped to
130 memory include remote files found in network shares or those located in
131 certain filesystems. Even if the operating system does allow mapping remote
132 files to memory, I/O operations on the file will likely be cached and
133 delayed, thus making true memory sharing impossible.
135 This solution has the major advantages of being independent of any backend
136 API and of being simpler to interoperate with from non-Qt applications.
137 Since \l{QTemporaryFile} is a \l{QFile}, applications can use that class to
138 achieve clean-up semantics and to create unique shared memory segments too.
140 To achieve locking of the shared memory segment, applications will need to
141 deploy their own mechanisms. One way may be to use \l QLockFile. Another
142 and less costly solution is to use QBasicAtomicInteger or \c{std::atomic} in
143 a pre-determined offset in the segment itself. Higher-level locking
144 primitives may be available on some operating systems; for example, on
145 Linux, applications can set the "pshared" flag in the mutex attribute
146 passed to \c{pthread_mutex_create()} to indicate that the mutex resides in
147 a shared memory segment.
149 Be aware that the operating system will likely attempt to commit to
150 permanent storage any writes made to the shared memory. This may be desired
151 or it may be a performance penalty if the file itself was meant to be
152 temporary. In that case, applications should locate a RAM-backed
153 filesystem, such as \c{tmpfs} on Linux (see
154 QStorageInfo::fileSystemType()), or pass a flag to the native file-opening
155 function to inform the OS to avoid committing the contents to storage.
157 It is possible to use file-backed shared memory to communicate with
158 untrusted processes, in which case the application should exercise great
159 care. The files may be truncated/shrunk and cause applications accessing
160 memory beyond the file's size to crash.
162 \section2 Linux hints on memory-mapped files
164 On modern Linux systems, while the \c{/tmp} directory is often a \c{tmpfs}
165 mount point, that is not a requirement. However, the \c{/dev/shm} directory
166 is required to be a \c{tmpfs} and exists for the very purpose of sharing
167 memory. Do note that it is world-readable and writable (like \c{/tmp} and
168 \c{/var/tmp}), so applications must be careful of the contents revealed
169 there. Another alternative is to use the XDG Runtime Directory (see
170 QStandardPaths::writableLocation() and \l{QStandardPaths::RuntimeLocation}),
171 which on Linux systems using systemd is a user-specific \c{tmpfs}.
173 An even more secure solution is to create a "memfd" using \c{memfd_create(2)}
174 and use interprocess communication to pass the file descriptor, like
175 \l{QDBusUnixFileDescriptor} or by letting the child process of a \l{QProcess}
176 inherit it. "memfds" can also be sealed against being shrunk, so they are
177 safe to be used when communicating with processes with a different privilege
180 \section2 FreeBSD hints on memory-mapped files
182 FreeBSD also has \c{memfd_create(2)} and can pass file descriptors to other
183 processes using the same techniques as Linux. It does not have temporary
184 filesystems mounted by default.
186 \section2 Windows hints on memory-mapped files
188 On Windows, the application can request the operating system avoid saving
189 the file's contents on permanent storage. This request is performed by
190 passing the \c{FILE_ATTRIBUTE_TEMPORARY} flag in the
191 \c{dwFlagsAndAttributes} parameter to the \c{CreateFile} Win32 function,
192 the \c{_O_SHORT_LIVED} flag to \c{_open()} low-level function, or by
193 including the modifier "T" to the
194 \c{fopen()} C runtime function.
196 There's also a flag to inform the operating system to delete the file when
197 the last handle to it is closed (\c{FILE_FLAG_DELETE_ON_CLOSE},
198 \c{_O_TEMPORARY}, and the "D" modifier), but do note that all processes
199 attempting to open the file must agree on using this flag or not using it. A
200 mismatch will likely cause a sharing violation and failure to open the file.
204 \page native-ipc-keys.html
205 \title Native IPC Keys
207 \keyword shared memory
208 \keyword system semaphore
210 \brief An overview of keys for QSharedMemory and QSystemSemaphore
212 The \l QSharedMemory and \l QSystemSemaphore classes identify their
213 resource using a system-wide identifier known as a "key". The low-level key
214 value as well as the key type are encapsulated in Qt using the \l
215 QNativeIpcKey class. That class also provides the proper means of
216 exchanging the key with other processes, by way of
217 QNativeIpcKey::toString() and QNativeIpcKey::fromString().
219 Qt currently supports three distinct backends for those two classes, which
220 match the values available in the \l{QNativeIpcKey::Type} enumeration.
222 \li POSIX Realtime extensions (IEEE 1003.1b, POSIX.1b)
223 \li X/Open System Interfaces (XSI) or System V (SVr4), though also now part of POSIX
224 \li Windows primitives
227 As the name indicates, the Windows primitives are only available on the
228 Windows operating system, where they are the default backend. The other two
229 are usually both available on Unix operating systems. The following table
230 provides an overview of typical availability since Qt 6.6:
233 \header \li Operating system \li POSIX \li System V \li Windows
234 \row \li Android \li \li \li
235 \row \li INTEGRITY \li \li \li
236 \row \li QNX \li Yes \li \li
237 \row \li \macos \li Yes \li Usually (1) \li
238 \row \li Other Apple OSes \li Yes \li \li
239 \row \li Other Unix systems \li Yes \li Yes \li
240 \row \li Windows \li Rarely (2) \li \li Yes
243 \note 1 Sandboxed \macos applications, which include all applications
244 distributed via the Apple App Store, may not use System V objects.
246 \note 2 Some GCC-compatible C runtimes on Windows provide POSIX-compatible
247 shared memory support, but this is rare. It is always absent with the
250 To determine whether a given key type is supported, applications should
251 call QSharedMemory::isKeyTypeSupported() and
252 QSystemSemaphore::isKeyTypeSupported().
254 QNativeIpcKey also provides support for compatibility with Qt applications
255 prior to its introduction. The following sections detail the limitations of
256 the backends, the contents of the string keys themselves, and
259 \section1 Cross-platform safe key format
261 QNativeIpcKey::setNativeKey() and QNativeIpcKey::nativeKey() handle the
262 low-level native key, which may be used with the native APIs and shared
263 with other, non-Qt processes (see below for the API). This format is not
264 usually cross-platform, so both QSharedMemory and QSystemSemaphore provide
265 a function to translate a cross-platform identifier string to the native
266 key: QSharedMemory::platformSafeKey() and
267 QSystemSemaphore::platformSafeKey().
269 The length of the cross-platform key on most platforms is the same as that
270 of a file name, but is severely limited on Apple platforms to only 30
271 usable bytes (be mindful of UTF-8 encoding if using characters outside the
272 US-ASCII range). The format of the key is also similar to that of a file
273 path component, meaning it should not contain any characters not allowed in
274 file names, in particular those that separate path components (slash and
275 backslash), with the exception of sandboxed applications on Apple operating
276 systems. The following are good examples of cross-platform keys: "myapp",
277 "org.example.myapp", "org.example.myapp-12345". Note that it is up to the
278 caller to prevent oversized keys, and to ensure that the key contains legal
279 characters on the respective platform. Qt will silently truncate keys that
282 \b{Apple sandbox limitations:} if the application is running inside of a
283 sandbox in an Apple operating system, the key must be in a very specific
284 format: \c {<application group identifier>/<custom identifier>}. Sandboxing
285 is implied for all applications distributed through the Apple App Store.
286 See Apple's documentation
287 \l{https://developer.apple.com/library/archive/documentation/Security/Conceptual/AppSandboxDesignGuide/AppSandboxInDepth/AppSandboxInDepth.html#//apple_ref/doc/uid/TP40011183-CH3-SW24}
288 {here} and \l{https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_security_application-groups}
289 {here} for more information, including how to obtain the application's group identifier.
291 \section1 Native key format
293 This section details the format of the native keys of the supported
296 \section3 POSIX Realtime
297 Native keys resemble file names and may contain any character that file
298 names do, except for a slash. POSIX requires the first character in the key
299 name to be a slash and leaves undetermined whether any additional slashes
300 are permitted. On most operating systems, the key length is the same as a
301 file name, but it is limited to 32 characters on Apple operating systems
302 (this includes the first slash and the terminating null, so only 30 usable
303 characters are possible).
305 The following are good examples of native POSIX keys: "/myapp",
306 "/org.example.myapp", "/org.example.myapp-12345".
308 QSharedMemory::platformSafeKey() and QSystemSemaphore::platformSafeKey()
309 simply prepend the slash. On Apple operating systems, they also truncate
310 the result to the available size.
313 Windows key types are NT
314 \l{https://learn.microsoft.com/en-us/windows/win32/sync/object-namespaces}{kernel
315 object names} and may be up to \c{MAX_PATH} (260) characters in length.
316 They look like relative paths (that is, they don't start with a backslash
317 or a drive letter), but unlike file names on Windows, they are
320 The following are good examples of native Windows keys: "myapp",
321 "org.example.myapp", "org.example.myapp-12345".
323 QSharedMemory::platformSafeKey() and QSystemSemaphore::platformSafeKey()
324 insert a prefix to disambiguate shared memory and system semaphores,
327 \section3 X/Open System Interfaces (XSI) / System V
328 System V keys take the form of the name of a file in the system, and thus
329 have the exact same limitations as file paths do. Both QSharedMemory and
330 QSystemSemaphore will create this file if it does not exist when creating
331 the object. If auto-removal is disabled, it may also be shared between
332 QSharedMemory and QSystemSemaphore without conflict and can be any extant
333 file (for example, it can be the process executable itself, see
334 QCoreApplication::applicationFilePath()). The path should be an absolute
335 one to avoid mistakes due to different current directories.
337 QSharedMemory::platformSafeKey() and QSystemSemaphore::platformSafeKey()
338 always return an absolute path. If the input was already absolute, they
339 will return their input unchanged. Otherwise, they will prepend a suitable
340 path where the application usually has permission to create files in.
344 Shared memory and system semaphore objects need to be created before use,
345 which is accomplished with QSharedMemory::create() or by passing
346 QSystemSemaphore::Create to the constructor, respectively.
348 On Unix systems, the Qt classes that created the object will be responsible
349 for cleaning up the object in question. Therefore, if the application with
350 that C++ object exits uncleanly (a crash, qFatal(), etc.), the object may
351 be left behind. If that happens, applications may fail to create the
352 object again and should instead attach to an existing one. For example, for
356 if (!shm.create(4096) && shm.error() == QSharedMemory::AlreadyExists)
360 Re-attaching to a QSystemSemaphore is probably unwise, as the token counter
361 in it is probably in an unknown state and therefore may lead to deadlocks.
363 \section3 POSIX Realtime
364 POSIX Realtime object ownership is patterned after files, in the sense that
365 they exist independent of any process using them or not. Qt is unable to
366 determine if the object is still in use, so auto-removal will remove it
367 even then, which will make attaching to the same object impossible but
368 otherwise not affecting existing attachments.
370 Prior to Qt 6.6, Qt never cleaned up POSIX Realtime objects, except on QNX.
372 \section3 X/Open System Interfaces (XSI) / System V
373 There are two resources managed by the Qt classes: the file the key refers
374 to and the object itself. QSharedMemory manages the object cooperatively:
375 the last attachment is responsible for removing the object itself and then
376 removing the key file. QSystemSemaphore will remove the object if and only
377 if it was passed QSystemSemaphore::Create; additionally, if it created the
378 key file, it will remove that too.
380 Since Qt 6.6, it is possible to ask either class not to clean up.
383 The operating system owns the object and will clean up after the last
384 handle to the object is closed.
386 \section1 Interoperability with old Qt applications
388 The QNativeIpcKey class was introduced in Qt 6.6. Prior to this version,
389 QSharedMemory and QSystemSemaphore backends were determined at the time of
390 Qt's own build. For Windows systems, it was always the Windows backend. For
391 Unix systems, it defaulted to the System V backend if the configuration
392 script determined it was available. If it was not available, it fell back
393 to the POSIX one. The POSIX backend could be explicitly
394 selected using the \c{-feature-ipc_posix} option to the Qt configure
395 script; if it was enabled, the \c{QT_POSIX_IPC} macro would be defined.
397 Qt 6.6 retains the configure script option but it no longer controls the
398 availability of the backends. Instead, it changes what
399 QNativeIpcKey::legacyDefaultTypeForOs() will return. Applications that need
400 to retain compatibility must use this key type exclusively to guarantee
403 The API in both QSharedMemory and QSystemSemaphore had the concept of a
404 cross-platform key, which is now deprecated in favor of using
405 QSharedMemory::legacyNativeKey() and QSystemSemaphore::legacyNativeKey().
406 Those two functions produce the same native key as the deprecated functions
407 did in prior versions. If the old code was for example:
410 QSharedMemory shm("org.example.myapplication");
411 QSystemSemaphore sem("org.example.myapplication");
414 It can be updated to be:
416 QSharedMemory shm(QSharedMemory::legacyNativeKey("org.example.myapplication"));
417 QSystemSemaphore sem(QSystemSemaphore::legacyNativeKey("org.example.myapplication"));
420 If the two applications exchanged native keys, there is no need to update
425 shm.setNativeKey(key);
428 Though if the older application did accept a native key, the new one may
429 opt to use \c{platformSafeKey()} with a second argument of
430 QNativeIpcKey::legacyDefaultTypeForOs().
432 \section3 X/Open System Interfaces (XSI) / System V
433 Never use existing files for QSharedMemory keys, as the old Qt application
434 may attempt to remove it. Instead, let QSharedMemory create it.
436 \section1 Interoperability with non-Qt applications
438 Interoperability with non-Qt applications is possible, with some limitations:
440 \li Creation of shared memory segments must not race
441 \li QSharedMemory support for locking the segment is unavailable
444 Communication with non-Qt applications must always be through the native
447 QSharedMemory always maps the entire segment to memory. The non-Qt
448 application may choose to only map a subset of it to memory, with no ill
451 \section3 POSIX Realtime
452 POSIX shared memory can be opened using
453 \l{https://pubs.opengroup.org/onlinepubs/9699919799/functions/shm_open.html}{shm_open()}
454 and POSIX system semaphores can be opened using
455 \l{https://pubs.opengroup.org/onlinepubs/9699919799/functions/sem_open.html}{sem_open()}.
457 Both of those functions take a \c name parameter that is the result of
458 QNativeIpcKey::nativeKey(), encoded for file names using
459 QFile::encodeName() / QFile::decodeName().
462 Windows shared memory objects can be opened using
463 \l{https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-createfilemappingw}{CreateFileMappingW}
464 and Windows system semaphore objects can be opened using
465 \l{https://learn.microsoft.com/en-us/windows/win32/api/synchapi/nf-synchapi-createsemaphorew}{CreateSemaphoreW}.
466 Despite the name of both functions starting with "Create", they are able
467 to attach to existing objects.
469 The \c lpName parameter to those functions is the result of
470 QNativeIpcKey::nativeKey(), without transformation.
472 If the foreign application uses the non-Unicode version of those functions
473 (ending in "A"), the name can be converted to and from 8-bit using QString.
475 \section3 X/Open System Interfaces (XSI) / System V
476 System V shared memory can be obtained using
477 \l{https://pubs.opengroup.org/onlinepubs/9699919799/functions/shmget.html}{shmget()}
478 and System V system semaphores can be obtained using
479 \l{https://pubs.opengroup.org/onlinepubs/9699919799/functions/semget.html}{semget()}.
481 The \c{key} parameter to either of those functions is the result of the
482 \l{https://pubs.opengroup.org/onlinepubs/9699919799/functions/ftok.html}{ftok()}
483 function when passed the file name obtained from QNativeIpcKey::nativeKey()
484 with an \c id of 81 or 0x51 (the ASCII capital letter 'Q').
486 System V semaphore objects may contain multiple semaphores, but
487 QSystemSemaphore only uses the first one (number 0 for \c{sem_num}).
489 Both QSharedMemory and QSystemSemaphore default to removing the object
490 using the \c{IPC_RMID} operation to \c{shmctl()} and \c{semctl()}
491 respectively if they are the last attachment.
1.10.0